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PREFACE , 



TheTeginning Te'acher Evaluation Study '(SIySTT^ project 

- ' • r 1 / " 

of the California Commission for Teacher Preparation and Lieensing>^he 

, ' ' / ' v / \ 

Commission is responsible for licensing teaehorsf in California and is 

trying to determine ^hat factors should be considered .in this pr6cess. , 

The second phase of the s£udy was cojiduet4d/o> Educational Testing 

Service for the Commission. Phase IT was the / \ ypothasee-'generatii^g and , 

instrument-development ptatse of BTE§, ( ETS had 'two tasks: (1)< to ;develop 

* i ' ' * '* * * ' 
'an assessment systeto\.tB measure both teacher and pupil 'behaviors* as /well 

j • , 

as other factors which might be related to these behaviors; and (2) to 
generate hypotheses aboufc the interrelationships between teacher and ■ / 

pupil behaviors and related factors. u < / 

' «. * • .* ' 

The, study was conducted in 43' schoqls. in eight districts throughout 

' ! ,\ ; ° ■ - - ^ " • >. . ' ^ " . 

the state of California^ A* total qf ^l\secdnd .grade* teachers ai^d 54 

< — -W ' . ^ . < O ' " V' ' * 

Ufttr grade t^achers^artiGipated in the*project during Phase IJ. 

* * ■,. v ' * ' , . . ' * 

. The iina,L report for Phase. II 'consists of "Several vblmes'J Volume I 

described the, design apd fatifenale* for the experimental design in'd 

' ' • r : ' . . : ' < ■ . H - i ' \ ' j 

'analysis ptQcedures. and,^ includes 'the major findings of. Phase' 71. Volume- 

II describes the conduct* of £he field study and the sample, of participant 

> ' Beciuse of 'the comjJle^/natiire 'of Phase II, a Variety "of techniques 

x P ' - " — ^ " v • " / - ' V • - «■ 

was oisedl to jhepsure' teacher7and puf>il behaviors- They are described in 

i ' « - . ' *> - / * 

Volumes illV'lV, and V. Results ? are* also ipeluded in ^these volumes* 

Volume III describes the observation systems in. detail arid is av&il£b 

- , • ^ . /.,'-.'- • ' ; x ; ! S •'' 

in thre'e separate^ipund sections. Ihe firs't secti-6^, Volume, II HI. 
describe^ the behavior recording, observation systeW used in. the proj-ect*-* 



1.2/ describes, ^the .category' syste|h used to, observe classroom actlvities- 



APPLE (Anectdotal Process to\t Promoting the Le^rnirlg Experience). -Volume 



RAMOS (Reading and Mathematics Observation System). The third section of 
'thi^v volume, III. 3., covers the videotaping of instructional activities* 
during vreacjing and mathematics. 

Volume IV concerns other aspects of -the measurement system and 
covers, both the pupil and teacher test batteries. 

, % The^ fifth volume covers a series of small studies done as part of 
Phase II. Volume V.l; looks at teacher aptitudes as related to teacher v *, 
behaviors. Volume V.2. i'$ Concerned with the relationship between 
teacher expectations and pupil performance. Volume V.3. reviews performance 
o*f pupils in the BTES tethers* classrooms for two years prior to Phase 
,11,, the historical test /aata. Volume V.4. discusses the Diagnostic Film 
Test, 3 device designedfjto assess teachers' skills in diagnosing reading 
problems and prescribing corrective action. Volume y.5. summarizes the 
results of work diaries/; completed, by the teachers Qn their reading -and 
athematics instructional program,* 1 * ' r ^ 

' Infonnatipn on tjhe. availability of these volumes can be obtained, /* 

/ • / -,. " - 

fxom: , - . / , 

\\ ' - ~ ■ 1 

* f pr . Frederick J. McDonald , / \ f 

s ' |EducatiQnal Studies * * ' , 

[Educational Testing Service s * * ^ 

I - /^Princeton, NJ 08540' * * - ' . t #S? 

, • ' ; F k ' • < ' ' • /> « iWi 

Information on' other phases- of BTES can be obtained frQm: ' H ^ 

"OaliforniaVComitiission >for Teacher 

Preparation and Licensing f 
* * v *. 1020 0 Street / s. 

Sacrairiento^ XA 95814 
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APP:JE Obser/atior. Svsieir 



obiective of t;he cia^roon observations made in Phase II 



<3 

"of <he Beginning Teacher Evaluation °. T .udv (EJESy war> to specify the 

obsetvabl'e teachen and pupil behaviors which affect pupil's learning. 

The APPLE (Anecdotal Processing to Promote tbe Learning Exoerience)* 

observation procedures used in the BTES -project were developed by 

col \cctin2 natural •claS-rror observations in over fc 150 classrooms 

I " \ 

or ibout 2000 elementary srt&agi _chi lcren over a three year period. 

The observations* obtained during ,this developmental periocL.were 

' - ' ' x \ \ 

Used to d&fine categories of observable pupil and teacher behaviors* 

bas*ed *o$* actual classroom occurrences. Rather than ^royide observer^ 

with categories or lists 6f behaviors developed 6n a«J priori basis, 

,we developed lexicons s fcrj teacKer and pupil behav^Cr on the basis 

'of the events actually^ observed in the clcts-srooms. The lexicon for 

Observation, in the Schools (Lambert and Hartspugh> 1971; Lambertv ' 

Hartsou^ Caffrey and ,Urbanski , 19J6) defines the universe of - 

observable" pupil and teacher behavior which characterized the Stress 

of ScKool Project 'classrooms . / Theism classrooms were' heterogenous 

.with- respect to urbanization, social status, .ethnic status and 

cl^sroom organization variables. 

^ The lexicon is an open rather than a closed system or ^ 

classifying- observable behavior. *-T#e lexicon can be au^e^ed by 

the addition of new entries within a, category or with new categories 

* * ■ 

at any time that an observable set of behaviors with similar \ 
characteristics cannot* be adequately assigned -an existing lexicon \ 



t*The system was originally developed in the Stress of School Project 
supported by NIMH Grant No. MH 14.605, 1968-7J. 



term or where new categories of observable .behavior . are -required . 

A Vcofeputer name" 'is assigned to the hew lexicon entry, by following 

rules for^machine sensibility* and the entry is defined operationally 

with examples of actual behaviors observecf in schools. A*B the lexicon 

expands ,^ it will ultimately define all observable school behaviors 

and the^addition* of ^pew entries will rarely be needed. 

^ The^ open-ended nature of *he APPLE naturalistic ^ystem made it 

ideally suited to the, observational' task' for the jj^TES -project; »• Rather 

than testing the significant relationship to achievement of a 

relatively fetnall numbet of predetermined Observational categories 

such as tho'se. defined in most_ category systems, all of the observable 

pupil and teacher events recorded by, the APPLE observers in the 

classrooms of the BTES teachers could be Examined for their 

relationship^ to school achievement. Therefore, a maximum amount 
• • * 

of information was .recorded, each item of which was available -for 
analysis in terms of its relationship to 'the learning variables, * 

Description of the APPLE Observation- System ' - % , 

. The APPLE observation system (1) requires observers to ptovitfe * 
low inference description's of pupil, ajid teacher behavior, (2) specifies 
classroom prganisrationarl Strategies, (3) describes teacher-pupfF 
and pupil-pupil interactions, and (A) describes the content of *the 
instructional program: Thomas (1932) and Haeberle (1959) pointed- 
but that ia order for the observer to become an ^^icient recording 
instrument in a classroom the observations must focus oh overt, " 
visible acts rather than concepts or categories* which require inferences 

...» * », ^ 



-r. t?e part cf observer. Trie ^it^t o: the observer cr.-the setting 

in vhien re or sr.e is observing must also be miR^sized (^rall,- l$p9), 
.and- the unit of behavior whicr is to.be recorded ^.ust^e defined. 

fiddle f 196 7) and Prail suggested that investigators specif y -whether • 

the .observers are to record* the specific Sehavlors of a single child 
.under a variety :*f circumstances a nur ^ er °^ children in • 

similar circumstances. s In- either case, t^e} cor&idered the selection 
^rfj bejhavibr, in naturally occurring units ratner £han on the basis of 

tir.e Sampler tc be the mere desirable schedule %lr observation. 
In the developmental work for the APPLE systfeih, we required 

observers to record their observations in« "descriptive, non- inferential 

term^, "and we infetructep then on methods of describing complete 

i • ..... 

segments of a pupil's bghavior* -These s&gnents or units ot pupils' 

J * , * 

behavior are called ''events'V^in the APPLE observational system. 
' «- * • * ' • 

After several thousand events were recorded, Ve began 'the task of 

assigning descriptive l&be^Ls to ea^h events These descriptive labels 

* \ - ' 1 1 " • ' * : " " 

designated entries in the preliminary lexicon for pbserved behavior. 

We also required that the qb'servers describe thje teacher initiating,,, 

behavior* an2. the teacher response to each pupil event and not jufet 

observe teacher 'behavior in isolation 1 of and- irrespective of its 



) • ■ M 

>il behaviors. In this 



effect on or response to pupil behaviors. In this^way the observa- 
tion system' provided a basis £ 4 qr describing y the characteristics of i 
te&cher behavior 'in relation, to the. behavioral characteristics, of ■ mm 
pupils. Sincq the APPLE Inf ormation^Sys tern (Lambert, P. Coijverge, 
E. Converse. and ^artsou^gh, 4 1971) was designed tq 'analyse elements 
of a % classroom observation singly. oi; in combination with .other 



elements/ the* observation and computer retrieval ^procedures ensured , 

the feasibility of. analyzing the combinations of'pupil and/ teacher 

.* " ■ » 

behaviors which were of interest in a particular research or 

i '■ .* ' 

evaluation question. v t 

The concept of an event in tlie APPLE Observation System. Tfre 

tern event .has beep used by jtfther investigator svto define, a unit of 

behavior., . Biddle <1967) conceived of brehavior eveiifs as' critical ' 

incident^ or episodes requiring a record of the social context and 

envjrohnen tal forces in order to be understood. Flanagan's 

critical -incident technique (1959) requires a judgment, perhaps 

an inference, that an incident 'is "critical 1 ' for a child. 

Kowatrakul (1953) specified six categories of pupil-behavior events 

whi^fch were expected to occur in natural settings: task-oriented, 

social work-oriented, socia,l-f riendly , '^omentary withdrawal, intent 

on work in an academic area not assigned, and intent- on work in a 

non-academic area. , . 

( " Elduson (1966) conceived of events as acttial objserved happenings 

as well as phenomena which could be inferred, She included the less 

"objective" types of event phenomena iti PsyCHES (Psychiatric Case 
* > x 

» c » H 

History Event System) because - these were typical of psychiatric 
records, containing as they Bo, factual information, patient reports 
of symptoms, £nd clinical observations 'and interpretations of patient 
information. "J 

The APPLE system originally conceived of an event as 'including 
anything that happened to a- pupil, was observed <to happen, reported 
to happen, or any information which would be pertinent to the pupil 
in ^chool, especially the interaction of the pupil with his 



instructional program, his teachers arc hi: pc-- r^> . Cur work for 
• ihesStress cr School Project indicated .tnat observers were ir.osft/ likely 
record behavior vhich was academic-i*jellectuciy or jsocial-etnotional, 

which characterized a pupil's response co school, ;cr Vhich was 

d i ' ' ' 

descriptive cf his behavior ar.d a^p&^ar&nce . i 

The concept cf ar. ^venc was revised som&what after analyzing 

this preliminary work. - In the present concep tualizatiorv, the \ 

structure cf information which we call ar event is focused on-^hat 

an ^sfervld pupil is dcir.g. Tve: rs describe -what- the pupil was 

observed tc do and include statements about the antecedent conditions 

of tjje observed beh&vicr , the teacher responses to the student if 

any, and the- consequences of the behavior. The event in the APPLE 

system is written as a sentence and is accompanied by the antecedent/ 

consequence descriptions. 4 The accompanying information associated v " 

with the event is part of the data that the observer must record. 

By treating important antecedents and consequences of pupil events 

as' different' classifications o£ information, data from the APPLE 

observation syst£n can be analyzed as a multiple^ category observation 

system in which. the informational categories to be use<3^can be 

selected for a particular research objective. 

. Mandatory information ift the observation record . In the 

'development of £he APPLE Observation System, we coiisidered obser- 

\ \ • . - o 

v^tions tp^ave limited informational value, if they were reported 

' c - * 

in isolation of descriptions of th& setting in vhich they occurred, 

v* . , 

The design of tfie system and the computer retrieval options permit 

several types of mandatory informbtion^to be carried along with the 

pupil efent. The types of mandatory information which have been 

/ » \ . . . . 



considered relevant to pupil eudntfr are (1) the instructional 



acftxvities of? the. teacher, (2) [the particular learning activity of 
the student, (3) the ins trupttfonal context — « the relationship of .the 
teacher to the instructional /activity of the child (e/g., whether^ the 
pupil Is vdrking under the qirett supervision of the .teacher or 
whether he is working, independently) , (4) the response of the 

teacher to p.upil behavior/ (5) the location^of the event,' (6) the 

' / * 
reporter or observer of t/he event, (7) the duration of the teacher 

activity, . C8) the duration of the instructional, context ,' and (9) the 

duration of the pupil behavior.* 

We designed the APPLE sys'tem in order to firing into focus as 

many relevant features of the i classroom as possible for a-particular 

research or evaluat/ion question. The t^pes of mandatory information 

can vary. In 'the /Stress .of School Project ^e requested the observer 

I \ * 
'to report the learning activity of the child, the instructional 

context, the lo/ation of the' 



event, and the reporter of the £vent.' 



Teaching activities and durajtion of behaviors were carried in the 

annotation, /n specifying die observation porcedures for ttie BTES 

/ / I ' ; , \ 

project^ we ^elected teacher instructional activity (teacher activity^, 



I: 

II 



pupil learning activity (pu|pil activity), instructional context (context),,^ 
and beache/* response as th^ mandatory information for $ach pupil event. 
'Figure 1 presents a model of the major categories of dat^ used by 
the qjvserfcer in the BTES project. The figure indicates t^hat specific \ 
response* of teachers and| pupils are recorded within a particular • ■ | 
,* teaching/ activity and instructional cpntext. \ 
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in the Stress #f School Project;. Comp] 




8* 



The sele£tion of Mandatory information fox; the ,BTEJS observations 
e/mp loved observational categories for whicf/'we had developed It 

/ ■ J ■ 7 

lete lexicons fdl^ pupil .£v en ts 



/ ' 



and teacher re^ponseV, ./asL well as ,prelimipary l'ex icons for te#£her / 

arCtivity and pupil activity were developed in the, Stress of ScWoL , 

! " ///''/ ll * *- 

Project. The ofeser^aftioris for' the Stress /of; School Project had been, 



obtained during" the entire schotfi d^y. . t|iu£, <£he pupil activates ^nd 
teacher activities wh/ich ha$I been ^entifjied represented a inpre 



comprehensive set of entries thkrr would be expected 
observations, which focused on ch^ract^er^stics of pupil and teacher 

' . r "•• J // ■ " ' ' ' 

behavior occurring ih the classjro<im during reading and mathematics^ 
instruction. -\ ^ ' 

The task of developing a; lexicon for the'BTES data necessitated 

\ - \ . ' 1 * 7 ; : ' 

the'crareful^examination of 31 all of the pupil and teacher ^cti^^ties 
likely to occur* £V as reading and mathematics were taught 



it/ > WeNies^ 



esigned 



the .observation form so, that the .observer was required to write a 



\ 



V 



description of the teaqher and pupil activity alo 4 ijg with the pupil 

' < * ■ >• \ N " ; ' 

' - , " \ >' ' " . * * . V 

: jevent and 1 teacher response. This requirement*, for additional English. 

\ ' ' , • v\- . « \*' ' *-> \ 

language entr4.es rather than a sitigle word description or ^abel 



V 



yielded a cpmplete' record of the types 9f teacher Activities and 
the. specif ic$ d-f pupil learning activities which\ occurred Quiring \\ 

M - * 4 ' \ & \ - V 

the 200 days APPLE observations. -Fronf t£\epe records we^ developed 
additional entries for the leiicon based on the^ xdefecrintions of „ : 
actual fcbserved teacher and pupil activities in reading apd mathe- 




tics- in the BTES classrooms. To .the extent t5hat~ tWs?e Vlassrooms 

■ ; » t '- • ■ \ \ ■ 

repfcesen£,a population of classrooms a£ ^second an£ fi^fth grades, 
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the lexikdns are" comprehensive enough to define the range o: 

* , '4' ' " . <■ \ - \ v 

atics 



)f \teaQher, 

and pupil activities during reading a^nd mathematics instruction 

/*v \ 



• \ 



7 



f9 



I 



Summary and (description of major categories of observation* used 

i T \ '.• ' ■ ' 

in f BTES . In order to enable ' the reader cq understand the scope and 

/ I " J 

structure of the APPLE Observation System as it was used in trie BTES 

Project, a mote complete *kplica|ion of the five major j^ggorie^ . 

of information is presentei het?e. It should he again, poted that / 

£PLE is not jl "category" taring, system^'. However, to tiestfidbet /the 

"objectives of the BTE£ in defining teacher /pupi^L behavior f^levAnt/ 

* f i j / 4 / / * 7 / k ■ y • ^ / i 

to/ pupil ouitttnues, the variables, to be examined an th^s Aepott art 

tike lexicon items f h*n the {fvej'mayor sets of /nybndat'orif , infonfiation 

selected for use in BTES. >h^se five types of'mandatcjfry information 

ar\ as follows : ' 

• t \ Instructional Context . r As noted eatlier,j tftfe concept of an 
instructional context defines the relationship! of the teache-t to 
thev instructional activity of *the*pupil. There are six major < 
contexts, ranging from the teachef directing tlie activities of,. 
/ the entire, class to an individual s$ud«nt wor}cp-ng or playing «• 
independently of tHe teacher's direction ,aQd independently of; 
the othek children 7 iin the class. Table 1 presets the list of 
instructional context lexicon items. > 

v ' TeacheAActivi,ty « This category of variables foeu&es.pn "V 

r> ,\the instructional activities of the 'teacher*, ** and may be thought * 
6f as antecedent behavior -jin a continuu^i of observation variables* 
As thfe lexicon is Constructed , various teacher activities appear 
under the heading^X'Instructing" , "Organizing", "Preparing", 
"Assigning"', "Disdip\ine" , and "Miscellaneous* 1 . 'Examples $f 
"Instructing" beh^vior^ include "Answering quespieRS 1 ^ "Explaining 

\a lesson", or ^Introducing new material". T^able 2 delineates the* 
full range of teacher activities i * ' • * , 

Pupdl Events . This categqry focuses on the observed behavio 
of the pupil/ The^e^der should refer bacjj \to the discussion of 
the concept »of kn event in* the APPLE system to 'gain a more complete 
understanding of the nature and limits of puMl events. .Very 
basically a pu^il events ife wfyat the pupil. ip\ observed to do at 
a point in time. The complete lexicon of pupil events contained 
in Table 3 gives an idea of the large range off ptipil behaviors 4 
described by the APP^E system. | 
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Teacher Responses . Tt>is category describes the consequent 
teacher behavior to a pupil event*/ The lexicon includes descrip- 
tors which can'be subsumed tinder three major headings: "Classroom 
Management .Strategies", "teacher Attitudes 11 j and "Teacher Communi- 
cations". 'Examples of teacher responses- ari '/Criticism' 1 , ^'Ignoring 1 ', 
and "Praise 11 . Tatfle 4 provides the APPLE Teacher Response Lexicon. 

' * fa ' i 

Pupil Activities . The Various learning and other activities 

-*#a .pupil engages in during the school day ate described! by the pupil J 

activity leixic^n. Some £xappl|e£ of, tread^g activities are "Phopics" 



and "Puactujation". -Examples oft mathematics activities 
and ''Gr/iphfLng 1 '., ^The, APPLE Lexicon of Pypjxl "Activities 



are "RiVisioti 11 

w , - - rrt— — fis pro$i<ied 

in TablJb 5,'J Pupil [Activities were used in this* study ftp sort' tjie 
* obserVafciohs ii^tb those' associ^t^d w with reading and^thpise associated 
witt} maxhefeatics instruction. , \ ? 1 $ • 
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* Observing the Duration* df Classroom Behavior 
& , 6 1 a — r-r* ' ' ' 

For some time we have considered the duration of riupil behavior 

i j *• <* 

to be an important classroom variable. Duration of beHatfLor can be 



defined] as the proportion of classroom time a particular behavior was - 

* - 1 

'observe^ .to occur or elqe as th^ duration of continuous pupil behavior/ 

!| r 4 ■ ^ 

thfexe are several ways in which to record the duration of behavior 
'in studies of pupil and teacher pe^orxnances. 

One can focus on the duration of pupil behavior, or one can record 
the duratio^ of teacher -behavior such as the, length of time teachers ' 

are involved in particular 'teaching activities. Sinfe the objectives 

< ' y 

of the BTES project wer%. to define teacher performance variables' in *' 
relation to reading and, mathematics pupil outcomes, we selected - 
_ ffYe~duT~a£ion of instructional contexts as the most ef f icien^way to 

appraise the' setting in whitfu instruction was offered and in turn to 

V 

^ estimate the effect of the length of time teachers were engaged in 

' * ' V 

L different patterns of teachec-pupil interactions.* (Refer 'to 

\ ' \ • 

Table 1 for the list of contexts.) The extent to whid) teachers 
eTfect different patterns of teacher-directed activity in reading 4^ 

v x \ • \ . ; 

and mathematics has been found to vary over classrooms. For example, 
some teachers conduct their reading classes almost 'entirely in a 
teacher-group^ context while, others rely\heav^'ly on independent- , - 

individualized assignments. Therefore a titap log of the duration 

\ 



. • f ' ' . . d l\\ 

j *lf the observation ta§k had been based in, high' schools where the 

context of instruction trarely changes in an instructional period,' 

time logs of the duration of differing Coiitextk might not be as 

. J ,_ useful data for <the study of teacher performance^ At the high 

m school level, the observation task might be more productive of 

crucial teacher data if time logSvWere b&secf on tin\e spent in' 

v * differing types of teacher activities. 



• \ 50 



ERIC 



23 



of ^rious .teaching" contexts appealed to be extremely relevant 

data., on the amount of teacher-directed learning activity, 

..*>•/« 

After §ach observation period, the APPLE observer completed & 

*• * 

summary report form on which he or she indicated the time each observe 

pupil spent in different instructional contexts, Iih addition, the 

learning tasks in which each pupil engaged and the materials -involved 

in the task w£re recorded. These summary report forms were used in 

I ] ; 

the detiriefing sessions with the teacher on the observation day. 

The teacher could make corrections in the information recorded, with 

pat^icular attention to those sections about activities and materials 

which the observer might not <have been able to complete correctly, 

Jxi the debrief ing*^ess ions the observer and teacher also reviewed 

the time in content en^^Les to see if they agreed with the teacher's 

plan for the day.. 

While we expected that t\e NN ltay the teacher administered the 

cl 



lassroom, in terms ,of the distribution of tinte in different contexts, 
would have an effect on iits^uctidn, w^also be^li^ved 'that different 

f^\mrab^y to one Vype of context 
e raasonaM.eness of this assump- 



types of pupils would respond m 
than another. In order to check the 



s • V \ \ 

tipn, we asked the observer to estimate f h^\proportion of ])upil time 



in, each context wjiich would be considered prochi^tive involv 



r \ment 



in 



the assigned learhing task. We difd not expect precite&^est images 

\ < A \ s * 

since some inference is involved ill making such judgments. However, 



ye did believe that the estimate or proportion of productive time 



^ in different instructional contexts would be a more accurate Estimate 
^han asking the observer to make an overall rating of productivity 
at\the end of the observation period, \ 



Procedures for Conducting the BTES APPLE Observations 

In the adaptation of the APPLE'* Observation System to the BTES * 
project, the BTES objectives were compared with thjp types of data which 
could be derived from the APPLE Observation Systenu We considered the 

\ * * • i " 

jteacher performance variables which our previous research suggested were 

1 " . ! ' * i 

Uikely to affect achievement and attitude dutcomes^ the procedures for 



ipbservation which were most likely to produce that kind of crucial 
information, and t hie methods of training to be employed in preparing new 



\ 

At 



observers for the\task. The identification of saliertt^f eatures of the 
APPl£ ObseW^ion System which were applicable t'p the BTES .objectives 
]ia¥"b"een made^in thep^ey^ious section and an overview of the APPL 



Observation System ha s* been provided. This portion, of the report de- 

V 

scribes the training activities and the jnethods by w^ich the obser- 
vations were obtained . — - \ 

■ " X" " 

P reparation of the .Observation Manual for the BTES Project \ 

Prior to the training sessions for the BTES observers* we prepared 
an APPLE observation manual for the BTES project. This manual include^ 
a description cjf the system, t^e goals of the observer? in theN^TES 
ptoject, the procedures for collecting observations, and guideline^ 
which defined the methods for making, recording and summarizing obsers 
vations. This document, " Instructions far APPLE Observers in the 

! % - \ 

Beginning Teacher Evaluation Study", \is reproduced in Appendix D of this 
report. In addition to the manual we prepared the APPLE Observation 
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Form and the Obse tVer Summary Report which are described below. 



2 



\ 



The Obser vation Forms- \ 

The APPLE. observation work f oyr the BTES t <jproj ect invo^jd J^wo form*'. 
The first was the APPLE Observation Form v (Fig[ire 2) on which theWserver 

described the pupil events and, instructional contexts ,along|with t$e' « 

1 . \ < ' - 1 . , • ' / ' " ' . • \ \ 

attendknt^ teacher activitiel, pupil activities and teachfcr * responses'^, . 

•r " * \ 

On this form the observer made a record of the time at which* the content 
changed for each observed pupil* The space^otr th'e f orm v f or, time* entries'; 
was to t?e filled i,n only -when there was a change 'in instructional . context 

The' Observer Summary Report (Figure 3)*' summarized £he dur.ation of dif fer- 

a I 

ent instructipnaj v cpi\texts for reading instruction and for mathematics 
instruction 1 . At ,thfe 'end of eact^ observation* period, ' the observer 
completed this form s using* ttte, contjext.4:ima\ entries W the -time 
designation entered ax the top wft'ety^each n^v' AI*PLE -Observation- Torm * 

The readet will note that a spkd-e ofi the^observation forn) provides 
for 'an observer rating, of t;he "quality of £upil behavior". We asked 
observers to rateas " + n pupil "behavior" which they., considered to be 
positive and appropriate for classroom expectancies.^ TKfey rated as 
^j- " only those behaviors which were inappropriate to the ongoing 
classroom activities/ M li there'was\no basis sV on wMch to malle a judgment 

of positive' or negative'^ipil fcehavidr, the observer recorded a "0". 

\ \ * * " 

We t recognized^that adding^ this rating t^o low inference observation 

procedure^ might introduce unreliable and less objective data to the 

observations^. ' We nevertheless included .these ratings in an effort to 

incorporate a^; much o| the observer's experience in the classroom as 



s possible. 



\ 



The observers majie their records continuously on the APPBE Obser- 
vation Forms during t^ observation periods — >In some classrooms reading 




\ 53 



Figure 2 
APPLE Observation Form 
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x^bserver Summarv Report 
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and mathematics instruction were carried on simultaneously.* More 
typicdj/ly, teachers ha\i specific periods tor reading £hd mathematics 
lemons . 

The Observer Summary Report detailed the time allocation in the - 
different instructional contexts separately by reading and mathematics. 
In "open" or "individualized" classrooms, observers differentiated 
between reading and mathematics instruction by using the time entries on 
the observation form and the descriptions of the pupil activity. The 
Observer Summary Report provides spaces for the observer to enter the 
pupil activity for each context and the materials which were being used. 
The caption "Focus of learning activity" was completed with the teacher 
in order to specify as precisely as possible the objectives for the 
instruction and to use this information to correct any observer mis- 
apprehensions about their records of pupil activity. 

„ i 

At the completion of each day s observations, the observer entered' 
the code number of qach pupil on the APPLE Observation Forim^jin^ 
viewed .the completed English language descriptions of pupil events, 
teacher activity and teacher responses. These materials, in addition to 
the two completed Observer Summary Reports, were sent to the Berkeley 
office of Educational Testing Service and transferred to the University 
of California at Berkeley where the observations were encoded and 
prepared for data processing. , 



* Ordinarily these simultaneous pupil activities would make observations 
very difficult. In this*'study, any difficulty ttas resolved by observing 
specific pupils who had been -chosen for observation p^ior to the actual 
observation. Thus, whether the reading or mathematics instruction was 
observed depended on which pupils were being observed. The methods for 
selecting the pupils to be observed are described later in this chapter. 
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( • . . ' 

Observ er Tr aini ng Act ivities f " • , 

With only a few exceptions, the, APPLE observations were 'made by 
observers new to the activity. We selected observers on the basis of 
prior experience in' school settings^as classroom aides, as graduate- 
students in education and 'psVcJjp logy with field experience in schools, 
or as teacher education student's. We selected. observers who were likely 
to have no heaa/v investment in particular types of reading and ma&he- 
waticb instructional methods. In addition to these selection criteria^; 
we considered the .general demeanor and appearance of the ^observers* arid 
the inferences likely to be -made about their neutrality and. obj ett ivity . 
Since an- observer was to make only one observation in some classrooms, 
his or her ability to a fit in" With the school ana 1 the classroom was an 
important factor in minimizing potent/Lai classroom disruption becaiise^of 
the observation activity. All potential observers were v interviewed by 
ETS professional staff and the authors of this chapter. j 

• The -selection process was intended to identify observers who pre;-* 
sented an unbiased attitude toward teachers and pupils,, who seemed able 
to handle communicat iort* cris'es and scheduling breakdowns, and who could 
cope with eventual unexpected occurrences such as teachers misplacing 
> the scheduling information about observations, or observers arriving for 
observation on a day when there was a substitute teacher. 

The training sessions involved several steps. Over an ^initial 
three day period, we met with ob^e^vers to review the "Instructions for 
APPLE Observers in the Beginning Teacher .Evaluation Study" and the two 
APPLE forms and to discuss the methods for t^te. observations . We had 
copies of the BTES project goals at hand to describe tt^ purpose o£ the 

.57 . N • 



observation procedures and' how the observations related to the total 
study*. ,Ve reviewed t^ese materials with the observers, went through 

w ' ' ' S # 

e^amplfc^ of observation records, and scheduled actual practice obser- 
vation sessions in elementary school classrooms whiqh were not in the 
BTES sample. In these training sessions we paired-off an experienced 
observer and a- trainee in several second and fifth grade classrooms. 
On the initial observation day, the trainee was required. to explain 

* y 

the purpose of the observations to the principal and the teacher. 
Then the two observers selected six students 'from the volunteer class- 
room for the trial observations. They made records during reading and 
mathematics lessons and returned for a draining session at the local 
project office in the afternoon. In these afternoon sessions, we reviewed 
the manner in which -the trainee had explained the project, discussed 

any particular problems which might have occurred with the teachers, 

\ 

and discussed the content of the observation records. This procedure 
was* repeated on a second, day and included another training review, 
session. • 14 

The "final training period centered' on^t?servat ion encoding. Even ■ 
though the observers would not be nea^ired to encode observations, we 
believed that it was important fJ^T them to understand the^methods which . 
were to t^e used so that their records would be completed properly. We 
continued to remind them that a good observation was one that could be 
encoded,- that is, one 'which was described clearly enough so that lexicon' 
na.mes could be assigned to the pupil events and the mandatory information. 

.Then all observers completed at least four 3TES observation days, 
after which we held a second group ' training session. At this time we 
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brought examples of their observations and pointed out instances where 
the information wdj> ft^mpletc jr ambiguous. Wv discussed scheduling 
and procedural problems in the schools, and made an effort fco prevent 
future difficulties by communicating our findings to the ETS Berkeley 
Office, where the system for scheduling the observations* was implemented, 

The group training sessions emphasized common kinds of observer 
problems. There was constant monitoring of each observer *s records as. 
they vere returned' for processing. Any incomplete records Were an- 
notated and returned to. the observer for resubmission. This procedure 
estaDlished a constant monitoring , and feedback system for each APPLE 
obseryer . 



Observat ion Schedules 

\ The schedule for observations followed the research design deve- 
loped for this project (see Volume I for the description of this design). 
Schools and teachers were informed of ^he observation days. APPLE 
observers received copies of. the • observation schedule and a list of the 
target pupils to be'obsetyed from the project office*in Berkeley. 

Th.e ETS. project team had 'established two patterns of observation 

< J " ' 

# S£h*duling* .The first W.as a fourr-day schedule .of paired observations 

for a 25 classroom subset of the BTE5 teachers selected to be demo- 

graphically representative. Four of the eight \observations • for each 

classroom were APPLE observations distributed throughout a two month 

• > * - ' ; \ 

period, -On £wo of the four days an APPLE observer was paired with a 

\ ■ ' > ' 

RAMOS obseryer; on a third day, two, APPLE observers Were present; and on 

- * '** • « ^ 

the fourth day, two RAMOS observers were in the classroom,* 



*The observation system RAMOS (Reading and Mathematics Observation' 
System) is described elsewhere in this report. 

- > . 50 
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*v Three observations were made in the remaining 70 classrooms.' 
\ * * 

Depeh4ing cn the schedule, we conducted .one or two days of APPLE 

observations in these 70 classrooms. Volume II, "Description of the 

Field Study and Sample", details the procedures for the observation 

*** . 

schedules and provides information about the way the schedule accom- 

modated the data collection for the two different observation systems 

At the conclusion of the APPLE observations, each classroom had 

been observed one, two or four times and within these sets of obser- 

» 

vacions there were 20 different classrooms which were observed simul- 
taneously by two APPLE observers. Although simultaneous APPLE 
observations had been planned for 25 BTES classrooms ? only 20 were 
actually observed concurrently. This departure from the original - 
plan occurred^because of difficulties encountered by some observers • 

in keeping scheduled appointments, largely as a consequence of the 
* * » * * 

fuel shortages experienced in California during this time 'period.^ 
These simultaneous observation days permitted an analysis of several 
possible sources of error in the observation system. Appendix A 
presents the results of an investigation of the reliability of the 
observational data for the BTES study. £ 

Identification of the Pupil Sample for Observation 
v. » ' 

We recognized that obs^vin^ all pupils w6uld not/ be appropriate 

*•''/- 

ror a short term observation schedule such as the one used in BTES. 

It was necessary to select a^uBset of pupils for observation who were, 

representative of the pupils assigned to each classroom. This subset 

t 

of pupils was called the target group. 

I 

Using Fall reading performance data and teacher rankings of / 
expected level of reading achievement, eight target pupils were selected 

60 
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in each classrccci, AjJ.1 pupils in each class were classified in the 



following matrix: 



-2 H 



u 



Achievement 
H L 



9 

ERLC 



The Median was used tc separate the data into high and low categories. 
The achievement data used were the total scores for reading and mathe- 

' matics on the California Achievement Test, The. sorts were made separately 
for reading and mathematics, but the target students were selected by 
using the reading matrix except where the data were not available. In 

"such cases mathematics d^ta were used, 

'The target pupils included one T5oy and' one girl each from the top, 
fron fust above the middle, from Just below the middle, and from the 
bottom ox thesis tribut ions iti each classroom. The target group inc^ucfod 
minority pupils in about the same proportion as in the total classroom. 
An alternate pupil was chosen for each .target pupil ^in the event the 
target pupil was absent. While there had to be alternate selection 
procedures in the absence of reading scores, or teacher rankings, the 
procedure described above was used to select mogt^frf the pupils.' 

' ' ' k N 

Observation Procedures for the Target Pupils 

In advance of the observation day, uspajiy the day before, each 

asked him or her to 



observer contacted the teacher to be observedd^* 



have available large pressure-sensitive labg¥%£o make a name tag for 
each pupil in the classroom. In case the teacher had forgptten to 
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provide these labels, the observers carried extra labels." In this w^y 
the target pupils could be identified without their knowing that they 

were the special 'object of the observer's attention. 

/ i 

The observers assumed a role in the classroom somewhere between 

participant observer and" "potjted palm 11 . ' Depending on the activities 

j 

being carried .on by the pupils and the teacher, the observer positioned 
himself or herself in the classrooiji for maximum visual access to the 
activities and behavior of the target pupils. s 



T6 do their joB£ the observers had to move int9 positions ere* 



Ltions 

thay could see and hear the target pupils, hear the teacher when he or 
she taltcbd to the pupil bei^ig" observed, and see the materials the pupil 

was using, when a pupil t6 be observed, was working independently, the 

/ 

observer had to move to a position from, which he or she could see what 

the pupil was doing. The rule was always t;o be in a position where the. 

! ■ , 

pupil could be seen or heard and yet be as unobtrusive as- possible. 

The observers were instructed ncrt to intrude in any way in any 
classroom activity. Pupils, however, view any adult present in» the 
class as a potential resource. On occasion, pupils asked the observer a 
question. The observers were instructed to answers very simple questions, 
mdny of vhich were social in character^ such as "Will you be here tomorrow? 
If the pupil asked a simple question about his assignment, for example, 



how to spell a simple word, the observer answered because this response' 
was the least disruptive. But the observer did not. remain to help the 
pupil. If the question was* more complicated or required ati explanation, 
the observer referred the pupil to the teacher. The observers reported 
that these content questions were relatively infrequent. Teachers did fc 
not complain about the observers interfering with instruction. 
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In general we tried to create a situation in which the natural 
\ « 

conditions' of the classroom were unchanged even though an observer was 

•*V - \ • 

present A The rule^of the observers had been explained" to the teachers; 

the observers were thoroughly instructed on their role. We trusted the 

\ v » ^ * i 

teachers to treat the observers as they^ usually treated other adults^ who 

\ 

came, into tr.eir classrooms. » 



Frequency of Observation Records' - 

- " > - 

^;TheVe were no fewer than three periods of observation during the 
school day. If the information was not available in advance, the 

observer consulted the teacher about the times when reading (usually- . 

i _> # 

two periods) and mathematics (usually one period) were taught. These 
periods, plus any other periods that the teacher designated as ones in 
which the observer could observe pupil and teacher behaviors associated 
with reading and mathematics, constituted the times for observation- 
Normally this amounted to four half-hour sessions a day, though there 
were some classrooms where observations were continuous over an entire 
morning^pr where the daily schedule in the classroom was otherwise 
different. 

The observation record describes pupil be.havior and teacher-pupil 
interaction as well as the instructional role of the teachet. On the 
observation form the observer noted the teacher activity wh^ch initiated 

\ ' ~ 

or accompanied ,the reading or mathematics lesson. \Typically the 



'observers simply described the teacher activity at beginning of v 
/an observation period, when the context changed or wh^n the* teaching 
activity within the context changed. These activities included explaining 
the X nstruct:i - ons for a lesson, organizing the lesson activity, working 
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with pupils on an assignment, conducting a discussibn,* lecturing], pr 
leading a question and answer session. (See Table 2 for the Lexicon 
of Teacher Activities.) 

After noting the general ^eacher activity and recording th| time 
of day, the observer made an observation record of the behavior! of each 
pupil on the list of target pupils, including in each observation the 
mandatory information for behavior events. With the exception faf 
context and Pupil Activity, .this mandatory information was recojrded in 
English language phrases or sentences, as was the description ojf the^ 



pupil event. Following this initial record of each pupil's behavior, 

the observer continued to make observations, usually no less than two 

for each five minute period, and as often as needed to provide a 

complete record of e^eh target pup il ! \^ behavior . Each ^ime the context 

changed, the observer made a record of the time "tlhe -new context began, 

and a record of each pupil's behavior in the new context. 

/ " ' 

Principles of Sampling Behavior to be Observed * • 

*The principle used to make- rules for sampling observable events is 

that each event has an equal Chance of being observed This principle 

means that an unbiased ^sample of events must be produced by the ob^ 

• servational processes used. Tftsre are two ways in which thi si principle 

may be, violated in making observations in the classroom, the observer 

may, through a personal bias, look at only those events which catch^is 

or her attention. The observer may fail to see an event because of the 

1 ( 

physical limitation of the observational setting* 

* * t 

k The first of these potential difficulties may be eliminated or 

ameliorated by using one or more of the. following procedures. The 

• • ' \ 
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observer is instructed to record every event occurring at a ^specif ic 1 

point in time, and is further instructed to make time-samples ,at Regular 

intervals. The result, is v a record of everything occurring^ at geVijpcfrc 

intervals which, if sufficiently elose together;, yi eld s< an, approximation 

of a continuous record. ' 

Another way to control for bias in the sample of events is to take 

place-samples, that is, to fcecord H everythin£ occurring in a locale 

within a larger environment. This procedure yields a different picture 

than a time-sample. * . . 

A third way to reduce bias is to record the actions of a person 

> n 

or persons. When this procedure is used the persons must be a 

representative sample of the persons in a larger set. 

«. * • 

Each methocf makes an assumption. The first method assumes- that 

• # *-» 

0 

time- samples represent the frequency with which the events actually 
occurred. Since events do not occur with equal frequency by units of 
time, the sampled' events approximate to an unknown degree the "true" 
frequencies. Investigators try to reduce this error by sampling as 
frequently as possible. . \ 

The second method assumes that events are distributed equally 
across simil'ar locates- The error in the observed sample of fcv^ents is 
a function of the differences between events within different locales. 

x The/ third method assumes that the actions of a subset of persons 
represents the domkin of events occurring in the larger collectivity. 
Amount of error is a function of the representativeness of t-he sample 
of persons' observed. * 

No one of these assumptions is more tenuous than the others, if 6ne 
has data on the domains being sampled. But the necessary information 
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required to describe a sampling v plan is not available for the first two 
methods. We do not have, for example, a continuous record of 'classroom 

\ ', 

events so that we can estimate how much bias is introduced by varying 

\ , 

the length of an observation unit ^pd the intervals between units. Nor 
do we know how events vary within similar locales in the classroom. 
^j g ^Th us , sakples of events which are produced by these two methods may 
-be biased and it is impossible v to estimate the bias. , v 

The third method, however, requires- only tjiat we have inforiha'tioli* 
on the group' to be observed, cbristruct.a sampling frame embodying this 
information, and then randomly sample within this frame individuals to 
be observed. Previous sections in this chapter described this sampling 
procedure for this study. APPLE observers are, therefore, providing a 
representative sample of pupil events. 

Two^ other features of the APPLE system should be pointed out. 
The observation provides a continuous record , not a sample , of the 
i nstructional context and €he teacher activity * Within instructional 
context and teacher activity, however, the observational method pro- 
vides a representative sample of teacher actions or responses as they 
occur with respect to the target pupils. The assumption is made ,that 
by selecting pupils representative of the class in terms oi specified 
characteristics, we sample representative teacher-pupil interaction 
dyads. ' This assumption is identical in character to that made about 
pupil events. But we do not know how this domain of 'teacher-pupil 
dyadic events varies as a function of the characteristics of pupils. 

We have provided v in Volume I, therefore, the rationale for * 
the sampling matrix used.- The logic of the rationale is that 
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teacher-pupil interactions are mort likely to var^|as a function of 
pupil characteristics used. Even if empirical data showed that such 
dyads varied more as a function of other characteristics, this study 
still provides a representative sample of teacher-pupil'interactions in 
the domain determined by variations in pupils' achievement and teachers' 
expectations. 

Pupil events a>e samp led* only in the sense that a representative 
* sample of pupils is observed. Teacher actions are sampled only in the * 

f> - -'mm 

sense that the,, teacher 1 s actions with respect to the representative 
sample of students is sampled,. But It should be remembered that 
teacher actions described by instructional context and teacher ' 
activity ate i^fttinuously recorded. . 

The other potential source of error in an observation system is 
observer error, the most important of which is failure'to observe 
events. The training of observers is designed to reduce this error, 
as are the rules for observing. Simplifying forms "and methods of 
recording are other ways of reducing the likelihood of such error. 

The ordinary test of the amount of this/ error is made by placing 

c J 

two or more observers in aekss, and then comparing their records to 
ass^ss^ their agreement. Asv^n/ted earlier, this procedure was used in 
• this study and its results are* reported in Appendix A. 

» « * 

Procedures, fo*r Observing Sampled Behavior 

An enormous number of discrete, specific responses or actions y 
sounds, and movements occur in a classroom. No' observer could record 
\ - all of these, nor should he. The observer's task is to note and observe, 

among all these potential observables, those likely to be effective 
stimuli if they were not£d v or observed by a teacher ar pupil. (An 

O 67 

ERIC • ' • 



40 . 

'effective stimulus is one which evokes a response..) Since the purpose 
.of this study ^is t6 find actions or resporfses which are effective 
stimuli, as few responses should be excluded as, possible. The observers 
were instructed what not to miss, and when in doubt, to record, 

j 

The observers were given the Lexicon for Observation in Scho61s- and" 
were a$ked to read through it. The, purpose of this rea-dirfg was to 
familiarize the observers with the kinds of events that occur in class 
rooms. These are the kinds of events feh^y were to^ record^ though'they 
were not^Lu aited ' to these types of events. The observers <fid not - 

- t * r *. . 

memorize the List. The goal was to get the "feel" Of the grange of / 
events which' they were likely to see and should record. The manual the 
^Observers used in the'field contained a list 6f the names of the entries 
in the lexicon. Jhe reader. ''shdfelft -und^starfd that obsepters did, not use 
these ev«it* names when recording £n the field. Tjie observers wrote 



.literal descriptions of wh^t they saw and. heard, ' % • • 

A ^ 7 

Almost. a'ny behavior of. tHe 'target pupil which was bb^erved could be 
recorded? but we instructed observers "to lopk especially for behaviors 
in the following areas: acatf^mic-intellectual, s&cial-emotional, general 
response to learning", and physical appearance. '(Refer to Table 3 for ' 
the entire set of pupi^ event 'flames found In the lexicon.) 

9 Each pupil behavior was accompanied by a, description of the teacher 
response. The Stress of School Project results showed that only abou|: 
one third of paturally observed pupil behavior is .accompanied by an 
identifiable teacher 'response. * We wanted all teacher response to pupil 
behavioV "to be, recorded for the BTEs/project. Therefor/ \$e required 



observers to pa# attention to non-verbal as as to verbal inter- 

actions with pupils. Non-verbal responses, such as "smiles,' 1 "notices" 
(without a -particular facial expression), "frowns"' or "ignores" were to 
be recorded if the observer was certain of the expression and if the 
reactions were in response to the pupil event. (See Table 4 for the 
entire set of teacher response names in the lexicon.) For some pupil 
events, however, recording a teacher response was inappropriate. In 
.these cases, the observer simply wrote "not applicable" or "none" or 
some the r statement describing the'absence of a teacher response in the 
space for the ,record of tjeacher response for the observation. 

Encoding APPLE 0bservati6ns 

All of the observations were encdded in the University -project 
office by three research assistants, each of,vhom had had considerable 
experiences in this work. 'The Tflain task in the encoding process is to 
assign a cqnP uter lexicon name to each mandatory entry. The first entry 
on the BTES APPl£ Observation Form is "Pupil Code". The code the 
observer- had entered was checked against the pupil code on the class 
list. | 

The computer name for "Event Name" was assigned by "refer ring to the 
lexicon. The event name is a label which describes the behavior, 
incident or information recorded by the observer. The event name is a 
highly condensed source of information whiq^ allows for convenient 
computer manipulation of the anecdotal material. As the reader' may have 
noted n the computer abbreviations for each item in. thfc pupil. evertt 



\ 

\ 
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lexicon appeared with the descriptors in Table 3 of the previous , sec tion. 
The Lexicon for Observations in the Schools (Lambert and Hartsough/ 1971; 
Lambert, Hartspugh, Caffrey and Urbanski, 1976) was always at* hand as 
the| encoding process continued: It contains definitions, examples of 
usage,' and sample observations collec ted "from the Stress of School Project 
The encoder made a decision about' which term to use as a 'label for 
the particular observation by comparing the content of the anecdote * . 
to the definitions and examples of usage for the pupil, event name 
or names which his coding experience'suggested might apply- Most 
single observations required no more than one pupil, event name., to 
characterise the content of the occurrence noted by Y the observer. 
In general, V>upil event names within the nine major categories of 
the pupil eveftt lexicon (see Table 3) were mutually exclusive in 
their application. However, the sets of codes across categories 

\ 

were hot necessarily treated as mutually exclusive items. "If the 
content of the observation described more than one facet of the 
pupil's behavior, fofc example, "Pupil talks incessantly to her 

neighbors but completes her subtraction worksheet with no errors", ' 

> 

theij more than one pupil event name was assigned and each was count;e<i 

a&'a separate tally in the compilation of frequencies of pupil event 

I 

names. In*the af brementioned example, the* observation* would be 
labeled both "-Talking" and "+Number Concepts", to 'general, however, 
most observations were written in such a way that only; one pupil ' J 
event name applied becayse only a single facet 6f the pupil's behavibr 
was observable at the moment. * 1 
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The next mandatory estr* is "pupil activity".* All entries in 

o • 

this f iel i were, crrvare I *ith the lexicon for oupJl activities (Table 5)- 
and the computer abbreviation entered into the appropriate space. - 4 
r The context field was to have been completed by the observer and the 

i •'*•-.. . , ' - ^ '** . . 

" encoder simply checked* to see if proper computer abbreviations were 
used. The lexicon namSS for "teacher activity" were compared with 
the ETnglish language descriptions and new entries ^pre added to the 
teacher activity lexicon when necessary. The computer name for the 
teacher aiti'-ity (Table 2) was entered next in the mandatory field.. The ^ 
final item of mandatory information was the "teacher response"*. The 
encoder compared the verbal descriptions with the lexic'on (Table 4) and * 
entered the computer name in -the mandatory field, • 

- When a set of observations had been encoded, each observation 
was defined Jby the, computer abbreviation^' for pupil code, event name, 
plipil activity, instructional context, teacher'*activity * and teacher 
response. A 1,1 of this information was keypunched and* became a line 

of computer information defining -orie pupil event. 4 „ , 

* \ * 
. % * - ^ T3Sfe eftcoder 'also checked the Observe^f^Summary Report against 

the Observation Forfrte for accuracy of the time entries for the different 

/ contexts The number of minutes each pupil- was. j udged to be productive 

« 

in each context was computed* from the information available on both 
forms. Thejse data then were prepared ff or keypunching. They provide 
evidence of "the.^time distribution of dif ferent. ins true tional contexts 

for each observed pupil, as velj<a~s the estimated' amount of productive^ 

, - ■ ' \ * - ( 

; t ime fcr^^ch 4 f\jpil . in the -observation periods for reading and 

^mathematics^ instruction. 
. ^ — _^ 




*In the space captioned "Subjection the GbseWatiyon Form, the observer' 
entered the .name of ^i^e ^target pupil. The nam^"was changed, to a code 
' number at .the ertd of tW observation day. > fc . 9 j 
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Preparation of APPLE Observations for Data Analysis 

At the conclusion of the observation pertod, we had collected 
thousands of observed events and approximately 400 s%miary reports 
on the time distributions of reading and of mathematics instructional 
contexts in the BTES classrooms. As soon as the APPLE Observation 
Forms and ttie Observer Summary Reports were encoded, they were key- * 
punched jaifd ready for processing into the APPLE Information System. 
The computer programs arranged the observations, by date of observa- 

s 

tion, within each pupil's file. Pupil files were arranged by code * . 
number within classroom units. When the computer files w£t;e * complete, 
they were ready for analysis by pupil chaf acteris tics. For the BTES <. ^ 
project our concern was to summarize the observation data using v the 
\classroom as 'the unit of analysis ' i*u4>tder to generate* classroom 

1 > ' V . 

variables for oui; s£udy of teacher performance predictors of achieve- 

■* » » 

ment and 'attitude outcomes. The preparation of these data using the 
classrooms as the uni£ of-. analyses required several steps. 

\ ' Specif ication'of the most frequently appearing observational data : 

' /* ' 

JDur first step in the data reduction was to have the computer make 

counts of the number of different types of ^upiT evertts, teacher 

'Ma • 
activities, instructional contexts- and teacher responses o The pupil 

activity information was used to sort-th£ observations associated 

* * » 

*.•*}'.• . . . » . 1 

with reading and mathematics instruction. It must be, noted that 

*\ • ' • - 

many of the BTES classrooms were not self -contained, and in still < 

other classrooms. several teachers integrated reading and mathematics 

>• ■ , , > * **** 

instruction into, single teaching sessions. The APPLE observation^ 

* t m " . < • 

and computer programs made it possible 'to define the reading and 

■i . , v 

mathematics observations on tfje basis of the actual learning -activity 



of the*pupil. (The pupil activities subsumed under the general instruc- 
tional .ireas of readir.e and mathematics are listed in Table 5.) 

We be^an the data 'reduction process by examining the raw frequency 

V 

founts from the entire observation record which includes reading, 
mathematics and c^her observed activities. These counts are found on 
Tables 6, 7, 8 and 9. We then obtained 'counts 'of * the most frequent 
pupil events associated with reading and ■ mathematics activities, ' 
eliminating from this step all 'events not observed in those lessons. 
We. followed the same procedure of 'making raw frequency counts for 
contexts, teacher activities, and teacher responses for reading. and 
mathematics. At this point in our work, /we had to establish arbitrar- 
ily a frequency cut-off point for inclusion of observation variables 
in the subsequent analyses. Our goal was to include the % maximum 
number of variables and yet have observation measures which occurred 
with sufficient frequency to serve as tae&ningful measures of class- 
room differences. We decided to include any observation variable, 

v , v 

if itsy raw frequency in either reading or mathematics instructional 

\ * 

\ » ~ o 

activities was 100 or greater. The list of pup 1,1 events , retained, £^ 
for_Xhe BTES analysis and their APPLE lexicon definitions aie presented 
on Tables 10 and 11, followed by frequency counts and definitions 
for contexts (Tables 12 and 13) > teacher activities (Tables 14 and 
15), and teacher responses (Tables 16 and 17)* 

Transformation of raw observation frequencies into comparable 
measures acVoss classrooms * Prior x to consideration of appropriate 
procedures for transforming the observational data into measurement 
units which were comparable, across all classrooms, ,we made a careful 
check of the representativeness of the target pupils. As described 



Table 6 

Frequency of Events Over All BTES Classrooms 



Greater -Than 100 
Fr equency Event Name 



Between 50-100 
Frequency Event Name 



3073 
1519 
1320 
1097 
962 
889 
871 
849 
555" 
464 ' 
453 
369 
' 349 
186 
■ 184 
165 
157 
155* 
145. 
138 
133. 
121 
108' 
106 



Engagement 
Work Habits 
Attentiveness 
Inattention . ■ 
Academic Performance 
Conduct 

Direction Following 
Number Concepts 
Talking .' . 
Oral Reading 
Questioning 
Participation 
Volunteering 
Phonic Skill 
Speaking 
Cooperation 
Inte'rests ■ 
.Reading Comprehension' 
Wandering 

Social Relationships, 
Pupil flelp 
Vocabulary 
Disruptive Conduct 
■Reading Vocabulary 



91 

7 ; 1 

67 ' 

61 

58 

54 

54 

52 

51 . 

51 



Egress,, 

Scheduling % 
Enthusiasm 
Responsibility 
Student Wish 
Restlessness 
Sitting Behaviar 
Distractibility 
Ability ' 
Activities 



c 
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Table 7 

"Frequencies of Contexts Associated with Events 
#(.)ver All ?TES Classrooms , 




Greater than 106 T f 
Frequency Context ^ 



Between 50-100 
Frequency Context 



3972 
3930 
3003 
2883. 
549 
474 . 
322 
280 
269 
143 



I - Group 
T - Class 
T - Group 
I - Class 4 
I - Individual 
Not Applicable 
A - Group 
Test , 

Transitional 
A - Class 



98 
82 



T - Individual 
A - Individual 
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Table 8 



Frequencies of Teacher Activities A^ciated 
with/Events Over All BTES ClassMoms 



pi- 



Greater 

Freauencv 



th^n 100 
Activitv 



5066 

gii 

301 
791 
783 
670 
568 
482 
>9 
433 
429 
413 
275 
264 
236 
225 
214 
200 
191 
178 
164 
154, 
106 



Work with Group 
o* Individual 
Asking 

Not Applicable 
Checking 
Circulating 
Helping 

Academic Organization 
Supervising 
Discussion 
Instruction Giving 
Question & Answer 
At Desk 
Explaining 
Listening 
Correcting , 

Testing Group or ClasSs 

Reading to Class or Group 

Dictating . ' 

At Board 

Answering 

Lecturing 

Distributing to Class 
Non-Academic Organizing 



t 

92 
78 
73 
61 
55 



Between 50-100 l 
quency Activity 



Around 

Directions to Class 
Out of Room 
Socializing with Adult 
Collecting Materials 




x Table 9 



\ 



Frequencies of Teacher Responses Associated 
withYvents Over Ml BTES Classrooms 



Greater than 100 
Frequency Response 



Between 50-100 
Frequency Response 



109D9 


(None) 


925 


Not Applicable 


894 


Positive Feedback 


313 


Redirection 


35 5 


Recognizing 


35? 


' Ignoring 


315 


Help 


307 


Praise 


255 v 


Instructing • 


209 


Questioning 


135 


Moves On 


. 126 


Correcting t 


125 


-Reprimanding \ 


124 


Explaining 



86 
69 
67 
54 



Permits 
Encouragement 
Acceptance 
Negative Feedback 
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77 
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able 11 



APPLE Lexicon Definition of Selected 
Pupil Events 



+ ACADEMIC 

•PERFORMANCE: 



Level of performance in school tasks; response to assign- 
meats or lessons. Correct. 



+ ATTENTIVENESS : 



Act of paying attention "to the subject being taught or 
discussed, as shown through physical orientation, ges- 
tures, verbal response. Used only in positive sense and 
in situation where teacher is instructing op leading 

lesson, 



+ DIRECTION- 
FOLLOWING: 



+ ENGAGEMENT: 



+ PARTICIPATION: 



+ VOLUNTEERING : 



WORK HABITS: 



CONDUCT: 



INATTENTION: 



TALKING: 



0 CONDUCT:, 



0 ENGAGEMENT: 



Behavior related to following directions given by the 
teacher. Cooperation with teacher directives, whether 
academic or behavioral . 

Act of being involved in subject of lesson. Used when 
pupil working independently of teacher, , , 

Description of an ax:t of joining in «a general class or 
group activitir. 

Any spontaneolus offering by \the child to do a Job or to 
give information, as in response J:o teacher ' s call for 
volunteers or to a question put to the entire group or 

class. 

n 

Positive manner or method of pursuing tasks, i.e. comple- 
tion of work, e/f iciency,, organization, neatness/ 



Negative school behavior wfyich cannot be placed under *a 
more specific event. . * ' - 

Not paying attention to subject being taught or dis- , 
cussed, as shown through orientation, verbal responses, 
gestures. Usually implies a long-term distraction from 
activity being directed by teacher. 

Inappropriate or forbidden speaking, such as speaking out 
of turn or during a quiet period. 

\ 

Neufral school behavior which cannot ba placed under a 
more specific event. 

Act of .being engaged in subject of lesson, but with less 
degree of involvement- or attention 'than + ENGAGEMENT. 
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Table 11 (Cont.) 



Not paying attention to subject being taught or dis- 
cussed. Usually implies a short-term distraction from 
activity being directed by teacher. ^ 



0 INATTENTION: 

0 WORK HABITS: 
* ORAL READING: 

+ PHONIC SKILLS: 



f READING 

COMPREHENSION: 

4- NUMBER CONCEPTS: 



- NUMBER CONCEPTS: 
0 NUMBER CONCEPTS: 



Neutral manner or method of pursuing tasks. 

Positive quality of students reading out loud; fluency*, 
accuracy, expressiveness, pacing, etc. 

Positive evidence of skill in sounding out new words; 
knowing the sound connected with letters. 



Pupil's correct understanding of t what he has read. 

Positive knowledge of numbers and number relationships. 
Includes identification^ of numerals, counting, skill 
with fractions, geometrical properties, etc. 

Absence of knowledge of numbers and number relationships, 
in -comparison with age and class level. 

Partial knowledge of numbers or number relationships (e.g* 
a concept partly understood) or absence* of knowledge of 
concepts more advanced than required by age or class level \ 
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Tablets . 

APPLE Lexicon Definition of* Selected 
^Contexts 



ADULT-GROUP: The class is divided into groups and an adult other than 

teacher is directing the, activity of the group in which the 
child being observed is participating. , 



INDEPENDENT- 
CLASS: 



INDEPENDENT- 
GROUP: 



The entire class is working or playing independently of the * 
teacher's direction. Teacher usually pre'sent but not direc- 
ts- _ x 



The group of students in which the child being observed is 
participating is working Or flaying independently of the 
-teacher's direction. 



INDEPENDENT- 
INDIVIDUAL: 



The student being observed is working, or playing independ- 
ently of the teacher's direction and also 1 independently of 
the rest of the clase. * 



TEACHER- 
CLASS: 



Teacher is directing the activity and the entire class is 
participating. V— 



TEACHER- 
GROUP: 



^The class is divided into, groups and the teacher is direct* 1 -, 

ing the activity of thfe group in which the child k^i&g ob- 
. served is .participating* t . M \/ 
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* Table 15 



APPLE Lexicon Definition of Seletted 
Teacher, Activities 



ACADEMIC 
ORGANIZING: 



Teacher getting class or group organized. m Giving directions, 
explaining and doing other activities which direct the stu- 
dents into activities.. 



ANSWERING: -Teacher answers question(s). May refer to a! single answer to 
a single question or to a- general , situation , where teacher is 
answering questions put by several students over a period of. 
time. ' • 



ASKING: 



Teacher is asking questions of a single student, a group, or 
ths. class, * - ' 



AT DESK: t Teacher is doing wotk at desk. Used when teacher's activity 

is unspecified or unknown and/or when she is available to 
* help students at her desk. 

m , 

AT BOARD: y Teacher is working at blackboard, either alone (e.g., writing 
* on board) or with student (s) . 



CHECKING : 



.CIRCULATING: 



DISCUSSION: 



EXPLAINING:* 



HELPING: 



INSTRUCTION 
GIVING: 



Teacher is checking work to see, if material is understood, to 
see if instructions are being followed, to see how well work 
is being done, or if work is correct. ' c 

Teacher is circulating^ around room, interacting wiUj^tu- 
dents, asking and answering questions, giving helpT^Tused « 
when a mo^ specific activity cannot be identified, or when 
several activities are happening one after the other, ■ too 
quickly to be specific . . 

Teacher is leading a discussion. Emphasis on student tes- 
ponse and comprehension, not simply on facts • 

Teacher is explaining something, *e«ither in response to a 
questiQn or, as part of a process of giving instructions. 

Teacher is helping a pupil or pupils in unspecified manner. 
Use$ when moire precise activity, such as explaining or ans- 
wering, cannot be x distinguished, or when several helping 
activities are happening too' quickly to be specified. * 



Teacher is giving instructions either about an assigned ac- 
tivity or about classroom organization. 
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Table 15 (Cont .)• 



LISTENING: 
I 



leather listens to a student, either dtiring oral reading or 



while student speaks to "her or asks? a question 



QUESTION AND 
AN'SWER: 



Teacher is asking a series of questions, which students ans- - 
wer/ Used primaf ily* in group or'class, context and distin- 
guished frdm ASKING by serial nature, of questions. ; 

* * * s * * • " " 

SUPERVISING: General situation „w£ere % teacher is .watching- over class^ or a, 
group. May occasionally -afek or answer 'cjuestion^ but gen- ' 
eral emphasis &s on observation rather than interaction * • 
with students. ■ ; • 

* 

rtoPvK;^ .VLIH: - Gei^ra^ unspecified instructional activity" <5f working with 
* \4n individual, a group or the class. Usetf -when no.other 

teaching * activity is specified. ■ . - , • „ 
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INSTRUCTING : 



DISCIPLINING: 



IEACnr>' HELP: 



IGNORING: 
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I.exiv.jr. Jt-fm^ti^n of Selected 
itacaer :\~ssonses 



A' 



J 



Teacher activities connected with her traditional jobs 'of 

mstrucy^gSid- supervising the everyda^ business of the 

c Ida s refer - "sfc^ a lessen," directed ,^<£^sj$laped, "called 

on -pupil," u^swWe'J a question. 



Pumshrent impjsea N or. cnild for his behav 
orinv l?ai 1 s office o: told to s«i t in corne 

i 

after school. 



or, B,g. .sent to 
r or/ to stay 



Teari.^r ^ives s^e specific instructional assistance to an* 
individual student. / 

Teacher seemingly awdre but not responding to behavior or ' 
performance of pup4-l. - ^ 



MOVES ON: Te'ashtr, Joes not comment on or *give feedback to : a pupil_ 

response, as in a question and answer session; instead, she 
calls on another pupil or moves on to another part of the 
lesson. Can occur when pupil's response is correct or in- 
correct . - ' , 



NEGATIVE 
FEEDBACK: 



NONE: 



Mild negative verbal reinforcement in response to academic 



h e m v i o r a I pe r f o r man c e 



e.g. , "wrong," "no."' 



No response to pupil performance or 'behavior. No distinp- 
tLon made between whether teacher aware of this performance 
% or behavior or not ^unless is clearly a situation^ of 

IGNORING) . ' ' 



POSITIVE 
FEEDBACK: 



PRAISE: 



QUESTIONINGS 



'9' 



RECOGNIZING: 



Use of mild positive verbal reinforcement in response ta 
academic or behavioral performance", e.g., "r^ght," "O.K.^" 
*"^bod, V "i ins," "correct." 



-Teacher, commends or tangibly rewards a student's activities 
or pro'duc ts . 4 
- 0 



Teacher responds to pupil 1 s [behavior or answer to a previ- 
ous question by herself ^asking a question; - ■ 



Teacher's neutral nonevaluative response to an academic 
performance or a nonacademic behaviour, e.g., "notices",, 
."acknowledges" ♦ \ 



REDIRECTING: A verbal or non^verba\L action ,by the teache'r to redirect a 
pupil's behavior. Done withjout apparent anger. 
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in a previous section, the target pupils werJ? identified from the 

classroom rosrters as 'being representative by sex, race,, and achievement \ 

level. We reviewed the observation' records for all observation days to 

* 

determine if any factors operated whith would , limit the opportunity to 
. observe all target pupils equally. f Careful examination of this potential 

* 1 At » 

source of bias in the records revealed that the target population was 
consistently observed in all classrooms and that there were no selective 
factors operating which would invalidate the observational data as 
being representative of classroom characteristics. 

Following an evaluatioa of the representativeness, of target pupils, 
the next step in data reduction was to tackle the problem of differing , 
frequencies of observations among classrooms resulting from variability 
in the number of observation^ (1, 2 or 4) . First, we examined obser- 

r 

yation rates for different , observers as a source of variability. As 
these were relatively uniform across observers, differences in 
frequencies of observation could then be attributed primarily to the 
. "observability" of thfe activities of the classroom. This^ conclusion 
was supported consistently by informal observer reports. For example, 

t w « , * * 

• ; 1 * 

in some structured classrooms the ^level of pupil activity was very low 
in comparison to a relatively unstructured classroom where pupils take 
responsibility for selecting their own learning 'activity. In the former^ 



case the instructional context In the classroom rarely changed.- Conse- 
quently, the observer could make only continuous observations 6f "pupil 
reading §ilently in his .reading book", or "ptrpll doing multiplication 
^problems in' his .workbook". In ' the latter case the observer would make 
many observations- since the context for the pup^l might change five 
times during the 'mathematics instruction as the teacher introduced the 

' \ 88 : ; . 
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mathematics lessen, the pupil completed ap assignment, tljie pupil 

vent ' to the derenstratien table and worked on other tqsksV and t^he pupil 

c * ? 

then returned to his desk—to begin an independent free time assignment , 
after which the teacher brought the class back to attention and they 
prepared to go out to lunch. 

\ k . 

Since we wished to obtain measures which would describe the class- 

r 

ro^TiS in our sample fairly, a method of' treating the raw observation 

■ f s 

frequencies had to be' "found which would" equalize fhenr f or differential 
observation rates due to variability in the* assigned number of observa- 
tiens in the classrooms *&nd/x>r to, differences in the extent' of classroom 
structure. Cyr solution was to prorate the total frequency of ea^h 
observation variable with respect to the total number of observations 
recorded overall (across all days of observation) for each teacher, \ 

\ 

after first separating the total recorld into subunits associated with\ 
/ * . ' 

reading and mathematics. .Thus, in actual fact, the prorating factor was* 
the % total number of observations during either reading or mathematics 
since all observation variables werd accumulated separately for these^ 
activities. These prorated Crequenyy measures provided us with classroom 
variables which, we believe, can be considered independent, both of 
number of days of observations and of* the possibly confounding classroom 
effects detailed above. 

The reader should be apprised that the types of observation records 
obtained j.n the APPLE Observation System and processed by the APPLE 
computer programs provide a very rich soured of information about pupils 
and teachers. The investigator has a wide variety of tfays in wtiich to 

*look at the observational* data*. Out decision to elect these methods of 

i * * * 

c^ata reduction was based on the objective of defining aspects of pupil 
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and teacher behavior to serve as classroom performance measures and then 
of analyzing the predictive validity of each of them. 

Reduction of Time Allocation Variables to Reflect Time Allocation by Classroom 

The basi£ data .for calculation of time allocation measures were 

taken from the Observer Summary Report which required the observer to 

record for each pupil a* nfeximum of the three most frequent types of 



reading contexts (r^, r^) and the three most frequent types of 

mathematics contexts (mp n^, m^)' in which the pupil had been observed 
during the specified- observation period. Also reported for each pupii 
were % the following: 

A = Total number of minutes in ep^&H s three contents 'for 
each pu^il 

B = Percent of total time devoted to each context for each 
pupil, e.g. 

B - 




C = 



D = 



Total number of minutes in each context during which 
pupil was engaged in productive, student-like behavior 
Percent of time in each context during which pupil was 
engaged ia productive, student-like behavior, e.e«. 



r = 
1 



r = 
2 • 



For, two types of variables^ the number of minutes and, the percent of 
total observed time,. scores for each teacher were -obtained by averaging 
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.-.urosi* observation days and across pupils in ,order s to obtain two sum3*d<^^/ 

a»-ro«*T. rr.ea^urcs, ^ne reporting a^oX^al* number of minutes (see Tables 

* » 

> >^ V, ;r. 8^ and '90, for examples'* and the other reporting the 
**> — - — , < 

averaged Drouorcion of the total time in the three contexts (see Tables 

*' *\ 

28 and 34, pp. 85 and 92, for examples). A^raging across observation 

* * *■ 

davs dvA across pupils was also done for the two additional variables, 

« • r 

number of productive minutes in each context .apd the proportion of 

: » total productive tire in contexts,. This prodded' two more summary 
• ' H ' 

.*>o.- u r rwisipis,, tfce average nunber of minutes or productive time 4 4 J 
f see Tables 27 .and' 33, pp. 84 and 91,, for examples) and the'average 
proportion of time which vas productive in each context t (see Tables 29 
and 35,* pp. 86* and 93 , f or 'examples) . 

In the review of the time allocation data , it became apparent ^, 
that percent of "time 'and percent of productive time measures woyld^ 
wive only* a partial picture of the ,total classroom instructional en- p 

virotununt. Ff the amount of .time the target pupil spent in each of his . 

# ■ 

tnree primary Contexts was not equivalent, or if all target pupils 
•w-r^ <t^u r md to the same contexts and„ spent different amounts of time 
u. each, then it was felt' that an additional metttbd for cpmputing the 
ii-*' allocation measures was needed. These supplementary measures pro- 
vided another way to look at classroom productivity. In explaining these 
* > is'ires, let use first illustrate the case when the number of minutes in 
riTt*>ts is similar across pupils. Take the case of three target pupils 
t.uh of whom spent approximately 10 minutes in Independent-Group contexts, 
but who rariifested productive student-like behavior to differing degrees: 
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\ 



Total Time 
in Context 



= 10 minutes 



T 9 - = 10 minutes . 



T 3 = 10 minutes 



Total Productive 
Minutes 

0 minutes 

10 minutes 

5 minutes 



Total- 

Gross = 30 minutes 
Time 



' Total" Gross 
Productive Tipse 
= 15' minutes 



Percent Productive 

Minutes 
- — » 

0% 
100% 
50% 

Average Pupil 
Percent Productive 
= 502 

Gross Per cent * " 
, Productive 
= 50% 



The -average percent of productive time would be: 0% +.100% -fy50% = 50^ 

3 ' 

The total time (gross number of minutes) in the Independent-Group 
context for the three target pupils was 30 minutes, the gross number ' ^ 
of minutes of productive time wa§ 15 minutes* and the percent of gross 
productive time was: 30 Minutes - 50% 

t 15 minutes . 

In another case the time spent in the Independent-Group -con teict 

varied ^among the target pupils. For example, consider the following case 



Total Time 
in Context 

= 5 minutes 

k = 15 minutes 



T^ = 25 minutes 



Total Productive 
Minutes 

5 minutes 

5 minutes 

5 minutes . 



Percent Productive 
Minutes 

100% 

33% 



20% 



Total 
Gross 
Time 



A'5 minutes 



Total Gross 
Productive^ Time 
- 15 minutes 



Average Pupil 
Percent Productive 
= 51? ' 

Gross Percent 
Productive 
33% 



When the percent of productive time was averaged across the three pupils, 
the average productive time for the classroom was- 51%, almost the same 
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tieeree of productive- in the first classroom. In the first example 

- rerccrr " -.-<=■ which* was productive (number of productive 

-inutes for all °v;: K> ~ diyaded by the" total number ,of* pupil minutes in 
the context)- v^S a l*/ 50" — the same value we found when we averaged 
percent of productive jtime for the three pupils.. In the second example 
:hc ;\<^ent < x - r " ^ ' II?^ which was productive was pnly 33% while, the 
average tfaiSie across' puoils was 51%. 



% 4 r .^u *t :\v 

r 

Tj tal --Time 
in Context 

T\ = 5 minutes 

i 

T ? « 15 minutes 

T\ = 25 minutes 



I *tal 

Gross- = 45 minutes 
Time 



. shows another way in* which average percent 
i percent productive gross time can vary: j 



Total Productive 
Minutes ; 

1 minutes 

, 5 minutes 

25 minutes 



Percent Productive 
Minutes , 



20% 
33% 
100%* 



Total Gjoss 
Productive Time 
= 31 minutes' 



Average Pupil 
Percent Productive 
= 51% 

Gross Percent 
Productive 
= 69% 



In this third -example , the gross -percent productive time was greater 
1 69V; than the average productive time across the target pupils (5i%).. 
• t lur see from these tx\nples that when pupils are more productive , in 
contexts that hive a terser duration, the gross percent productive time 

v * I 1 ■ 

context will be i^attr than the average percent of productive pupil^' 
minutes. 

The choice of vl u h value most fairly measures the classroom 
instructional impact will require further evaluation. However, for 
Larity and completeness of data presentation, two additional types of. 



9 



and percent of gross productive titoe in* context (the total number 'of 
minutes . the target pupils were judged to be productive divided by the 
total number of minutes across all target pupils in. that soirtext.O . 
(See Tables ,30 and 31, pp. 87 and 88 M fpr examples.)^ ' , \ 

At the conclusion of the data reduction phase we had .t^ie following^ 
data for each teacher available for analysis': 

—%\ - ■ ■ 

"17 Prorated incidences of the major ob^ervaripiral~xaregorie3 ~" 

a) Pupil events t 

■ * Jb) Instructional contexts • 

c) Teacher activities 

c * d) Teacher responses " * 



2) Time allocation measures for instructional contexts 

a) Averige number of minutes in each context 

b) Average proportion of total minutes in/each context 

c) Average number of productive minutes in each context 

d) Average proportion productive minutes in each context 

e) Average proportion gross number of minutes in each context ' 

f) Average proportion gross productive minutes in ea'ch context 



'•„ . , . • ,67 

0 • » 

> Results 

» 4 

The primacy function of,the analvsis of ,£h§ APPLE observational - • 

* * 
data for the Beginning Teacher Evaluation Study was to, provide testable 

hypotheses about the classroom performances of teachers . that ^are related" 

*to pupil growth in the acad,einic areas of reading and mathematics.. In,* 

order to reflect the interests of the Commission, we have systematically 



organized and analyzed the information to focus on teaching performances w 
.Each teacher's pupil^ were treated as a group. Although individual 
'teachers were the unit of analysis, the subset of students .who had been 
selected <® be observed represented a comprehensive' range of pupil 

attributes' ■ They were a Srepresentative "cross-section" of pupils in 

•» ■* ' 
BTES classrooms*.* / u . . . 

In this section»6f the repjDr t , -we* will describe the major findings 

from the APPLE observational -phase ^ » Thes^e findings will include, by 

.grade level, Nummary statistics which include descriptions of the average 

*s * * * *" - 

prorated^ frequencies of observation Variables, for reading and mathematics;, 
as well as summary statistics far the time allocation in instructional 
context variables. (For a detailed report of these descriptive statist » , 
ti^s compiled for BTES classrooms stratified by Socioeconomic Status $nd 
Geographic Location of School see Appendix 3.) Also- presented in this, 
section are predictive validity studies of each of th^\APPLE variables j . 

• ' / » • 

using end of year achievement and attitude scores' as the criteria,' 
controlling for Fall test performance. Finally, we will present /tabled 
of cross tabulations and describe some of the relationships among teacher 

' • \ : .• / v % : 

activities,* teacher responses, teacher classroom organizational fetrate- 
gies, and pupil aca'demic and attitudinal changes by^comparing the • classrooms 

• .. V -' ' ' •" 
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where the pupil grbwth was greatest to those classrooms where t&e least 
growth was produced. 

The findings to be reported below, with the exception of the 
crosstabulation tables, consist of statistics associated with each of 
the* fifty incidence variables and sixty time allocation variables ' - 
analyzed individually. When the APPLE variables' are subsequently ' 
analyzed along"with the other measures colle6ted during the BTES 
study y it is expected that they . will be subjected either to empirical 
data reduction or v to rational combinatorial procedures. Such data 
reduction seems mandatory *if the total number of variables is not to 
exceed tlie number oj classrooms, the unit of analysis for the studies- 
to be reported in Volume I. In an attempt to provide a framework • 
within which such d£t;a reduction might proceed, we have conducted 
factor analyses -within each of* the major* observational .categories . 
Because of the technical nature of this material, the results of 
these analyses, by grade level and instructional area,,, are given in 
Appendix C. Also, reported in Appendix are validity studies of the c 
relationship between factor scores derived from the* analyses and the 
enii of the*year achievement arid attitude outcomes, controlling for 
fall test- scores. "' * * > 

r ft ' imt ^ - / / 

Having 'thus chosen to analyze the APPLE data as individual 
variables rather" than as- grouped variables, we w£re-*#9£ed with a x . j c 
large array ^ of ^outcomes to be dpcribed. Confronted with this task, 
we have elected x in tfje following discussions of results, ^ to cotnment 
oh'only those relationships which seemed €o u's* especially noteworthy . 
;on-either' theoretical or pra'ct-itfal grounds, , The careful reader may 
note 'that some variables .starred as significant may not be mentioned 
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in the verbal description Of the results.' This is not an oversight 
but merely reflects our judgment about which "findings were most 
worthy of elaborated discussion. 

« Some readers also may be concerned to find small variations in 
the numbers of teachers included for the different analyses to h|£ ^ 
reported below. Although ninety-five teachers were originally scheduled 
/*f or" observation, one teacher subsequently couldn't participate because 
of injuries revived in an automobile accident. Accordingly, the 
incidence data reported in Tables 18 through 25 was based on N=94. 
Likewise, the tables of descriptive' statistics for instructional 
context time allocation variables show N r s which are different from 
the incidence variables and differ also between heading and mathe- 
matics. The loss of data for these analyses (N=90 for reading time 
allocation variables and N=84 for mathematics time allocation vari- 
ables) is a consequence of two observer 1 s initial confusion* about 
the nature of the recording task f or „the "Observer Summary Report 11 , 

4 

Although APPLE project staff were in routine contact with observers, 

inevitable delays in receipt of^ completed observation materials made 

it impossible to rectify the misunderstanding until several days 

observations had been completed. Consequently, the total N available 

for analysis was reduced as indicated. \ 

Grade Level Contrasts for Observations in Reacjing and Mathematics Instruction 

Our first set of descriptive information sunparizes the mean, median 
and standard deviation for each of the observation variables — pupil 
events, instructional contexts, teacher activities and teacher -responses. 
In nearly all cas^s the values for v means and medians indicate that the 

distributions of classroom observations were not normal but were posi- 

* »* > 

; '97 ' ... •< ." 
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Table 18 



Average Prorated Incidence of Events Observed 
During Reading in BTES Classrooms^ 



Events 



_ Grade 2 N- 41 
X Median S.D. 



Grade 5 N= 53 
Median S.D. 



+ Academic Performance 
+ AtDentiveness 
+ Direction Following 
+ Engagement 
+ Participation 
+ Volunteering 

+ Work Habits 

- Conduct 

- Inattention 

- Talking 
0 Conduct 
0 Engagement 



0 Inattention 
0 Work Habits 
+ Oral Reading 
+ Phonic Skill 



Reading Comprehension 




- 98 



^ 71 g 

f Table- 19 

Average Prorated Incidence of Events Observed 
During Mathematics in BTES Classrooms 



Events 



Grade I N= 41 

Indian S.D. 



Grade 5 N= 53 
Median S,D, 




+ Academic Performance 
+ Attentiveness 
+ Direction Following 
Engagement 

.pat ion 

+ Volunteering 
+ Work Habits 

- Conduct 

- Inattention 

- Talking 

0 Conduct 
0 Engagement 
0 Inattention 
0. Work Habits - 
+ Number Concepts 

- Number Concepts 
0 Number Concepts 



1 L 




2 79 


. 32 


.10 


.93 


< 70 
J • / u 


J • V J 


7 16 


6.41 


3.08 


8.00 


? 1 S 
t> ij 


t 

• ?1 


4.66 


1.68 


.33 


3.02 


11.96 


io io 


11.31 " 


19.25 


16.70 


14.30 


2.36 


h 


6.07 . 


1.16 


.14 


2.20 


2.15 


.80 


' 3.16 


1.74 


.31 


2.29 


7.57 


7.05 


8.37 


5.79 


4.38 


5.61 


1.19 


.21 


2.25 


1.30 


.17 


2.43 


4.30 


2.60 


5.15 


5.60 


4.5-3 


5.57 


3.01 


1.45 


4.81 


4.06 


1.85 


6.2# 


l.*61 


.31 


2.55 


1.58 


.28 


2.54 


1.60 


.29 


3.25 


1.16 


.16 


2.63 


1.63 


.23 


3.99 


2.37 


.34 


4 .'9 2 


. 3.15 


1.38 


4.13 


5.05 


3.30 


6/85 


13.41 


10.15 


12.47 


9.67 


8.18 


8/. 8 5 


3.75' 


2.28 


5.44 


2-. 06 


.27 


132 


2.85 


1.65 


5.97 


1.79 


.34 


2.54 
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tively skewed; that is, there was a piling up of frequencies at the 

lower end of the distribution. We believe that these distributions 

represent the actual distributions of the variables under the conditions 

of observation and that they are not artifadts of the procedure. 

Comparisons of average prorated incidence of pupil events by grade 

level and Instructional area * The average prorated incidences of.- pupil 

events observed in reading and mathematics instruction at the second 

and fifth grade -levels are presented on Tables 18 and 19. It appears 

that pupil behavior at these grade levels in both reading and mathematics 

is very similar. T here is more observable evidence of reading and mathe- 

inatics skills in second than in fifth grade as evidenced by the Jftgher 

rates of "+ Oral Reading" and "4- Number Concepts", as well as greater 

frequency of observations of "Phonic Skills", "Reading Comprehension", 

and neutral or Negative evidence of number concept skiils. "Plus (+) 

Engagement", the lexicoh term which is assigned to pupil events of 

involvement with a lesson on which the pupil is working independently of 

the teacher, is more frequently observed in mathematics than in reading 

at both second and fifth grade levels. Negative pupil behaviors^such as 

Conduct"^/- Inattention", and Talking" are observed with similar 

frequency in both subject matter areas. 

Comparisons of average prorated incidences of instruc tional contexts 
>» 1 ' ■ — ^— f — 

by grade level and instructional area . Reference to Tables 20 and 21 / 
illustrates the great differences in the wSy instruction is organized 
for reading and mathematics at second and fifth grade levels. The 1 
context in which the teacher is teaching the whole class simultaneously* 
is most frequent in the second grade for mathematics instruction, while 
"Independent-Class" (pupils wording independently , on the same assign- 



Table 20 7 

j 

Average Prorated Incidence of Contexts Observed 
During Reading in BTES Classrooms 





_ Grade 2 N= 


41 




Grade 5 • N= 


• 53 


Contexts 


X 


Median 


S. 


D. 


X 


Median 


S.D. 


Adult-Group 


7.4^ 




13. 


97 


1. 22 


.10 


4.39 


Independent Class 


7.28 


.38 


12. 


21 


19.49 


8.80 


25.66 


Independent Group 


28.87 


, 28.00 


17. 


44 


32.57 


31.35 


21.38 


Independent Individual 


5.08 


.19 


12. 


87 


3.05 


.16 


9. .63 


Teacher Class 


8.66 


1.60 


15. 


51 


12.19 


3.93 


18.23 


Teacher Group 


32.54 


33.68 


18. 


86 


24.63 


22.08 


22.02 



\ 



\ 
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Table 21 

Average Prorated Incidence of Contexts Observed 
During Mathematics in BTES Classrooms 





Grade 


2 N= 


: 41 


Grade 5 N= 


53 


Contexts 


X 


Median 


S.D. 


X 


Median 


S.D. 


Adult Group 


4.07 




31 


12.18 


.63 


.32 


4.57 


Independent Class 


13.06 


1 


48 


19.97 


27.11 


16.50 


30.37 * 


Independent Group 


22.41 


21 


20 


20.00 


22.49 


15;85 


24.81 


Independent Individual 


3.96 




39 


11.05 


.98 


.87 


4.23 . 


Teacher Class 


30.23 


21 


43 


31.61 


21.37 


8.28 


25.85 


Teacher Group 


14.04 




35 


21.26 


12.95 


1.08 


19.68 




£_ 
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menfc) is most freqjent for fifth frade Mathematics. Teachers tend to 
-rely more on whole class instruction and identical assignments for all 
pupils in mathematics than in reading. Some idea of the extent of 
individualized instruction can be inferred from the incidence of the 
"Independent-Individual" context.. It was one of the least frequently 
observed classroom contexts at either grade level. There are a greater' 
number of observations of this context in grade two than grade five and 
more in reading than in mathematics. 

Except for the "Independent-Individual" context we can conclude 
that fifth graders are expected to spend more of their classroom time 
working independently of the teacher (in groups and as a whole class) in 
Koth reading and mathematics. Although it is relatively infrequent at 
either grade level, group instruction by an adult other than the teacher 

is more frequently observed in, second gradfe classrooms in both reading 

\ 

and mathematics instruction. "Teacher-Group" instruction in reading is 
more common at second than fifth graded but this context is equally 
frequent at both levels for mathematics instruction. « - 
Comparisons of average prorated incidences of teacher activities 

by grade level and instructional areas . Tables 22 and 23 show central 

x > 

tendencies in teacher activities. Teachers at fif th^grade are *at their 
desk more often than are second* grade teachers. There* is a greater \ 
incidence of "Working With" at second grade, an activity reflecting v 
instructional work with groups, individual pupils, or the entire class. 
Teachers are at the board more frequently in mathematics and more fre- 
quently in second grade. Teachers circulate and check pupil work and 
are observed helping pupils more frequently in mathematics than in 
reading at both grade levels. T ; 
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Table 22 ' 

Average Prorated Incidence v of Teacher Activities 
Observed* During Reading in BTES Classrooms 



Grade 2 N= 41 . Grade 5 N=53 



Activities > 


X 




Median 


S.D. 


X 


Median 


S 


• D. 


Academic Organizing 


3. 


05 


1 


90 


3. 


96 


2.49 


.18 


4 


.93 


Answering^ \ ] • 




66 




.04 


2. 


84 


3.45 


.33 


10 


.84 


Asking [' 


6. 


46 


4 


.35 


8. 


47 


4.78 


2.33 


5 


.78 


At Desk ■ 




40 




07 


1. 


68 


5.65 


.20 


14 


.21 


At Board 


1. 


58 




.07 


6. 




.89 


• .07 


2 


.57 


Checking 


• 2. 


62' 




.14 


4. 


86 


4.85 


1.05 


9 


.97 


Circulating 


5 


12 




.14 


16. 


20 


7.21 


.29 


15 


.21 


Discussipn 


1 


73 




.12 


. 4. 


75 


" 4.07 


.26 


8 


.8* 


Explaining' 




48 




.11 


1. 


P8 


1.42 


.10 


3 


.90 


Helping « 


3. 


64 




.19 


6. 


88 


V— ^.68 


.45 


6 


.62 


Instruction Giving 


2. 


30" 




.93 


3. 


60 


, 3.38 


1.58 


' 5 


.22 


Listening^ 


2. 


50 


1 


.23 


3. 


54 


1.70 


.33 


' 3 


.78 


.Question ■& Answer 


1. 


42 




.13 


2. 


77 


4.16 


.39 


8 


.90 


Supervising 


1. 


50. 




.10 


4. 


66 


2.83 


.28 


6 


.93 


Working With 


46. 


58 


50 


.90 


26. 


96 


31.75 


27.20 


25 


.08 



V 



A 



6 
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Table 23 

/ 

Average Ptorated Incidence of Jeacher Aptivities Observed 
During Mathematics in BTES Classrooms 



\ Grade 2 N= 41 ~ Grade 5 N= 53 



Activities 




XX 


Median 


S. 


D. 


X 


Median 


.S.D. 


Academic Organizing 


—j 


2. 


03\ 


.31 


3. 


77 


1.65 


.50 - 


2.86 " 


Answering 




1. 


41 


V .08 


6. 


10 


2.83 


.09 


8.60 


Asking 




5. 


39 


\.00 


8. 


07 


4.12 


.27 


6.72 


At Desk 






45 • 




2. 


31 


5.93 


.22 


14.59 


At Board 




4. 


86 


■ - h9 


9. 


78 


1.62 


:16 


3.10 


Checking 




6. 


69 


1.41 


11. 


45 


. 6.98 


2.06 


.11.89 


Circulating 




12. 


28 


1.48 


23\ 


03 


8-. 23 


2.00 


12.30 


Discussion 




2. 


40 


.16 


13. 


68 


. ,71 


a 8 


1.98 


Explaining 




2: 


10. 


.35 ' 


4. 


53 


4.54 


* .54- 


11.02 


Helping 


* 


5. 


13 


.31 


9. 


68 


7.38 


1.88 


10^0 


Instruction Givdng 




2. 


93 


.95 


4. 


40 


3.25 


i 


8.17 


Listening 






.16 


.03 




72 


" .15 


.07 


- .75 


Question £ Answer 




1. 


51 


.12 


4. 


11 


i.6& 


. -17 


5.05 


Supervising" 




2. 


69 


.39 


. 5. 


35 


2.52- 


.10* . 


N 7.58 


Working With 




29. 


37 


22.88 


25. 


56 


22.79 


17.78 


21.71-' 
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Comparisons of average prorated incidences of teacher "responses 
by , grade level and instructional areas . Thje quality of ^teacher responses 

.to pupil events is very similar across both grade and instructional 
areas (Tables 24 and 25). About two-thirds of the time there is no 
teacher response to an observed pupil event. "Positive Feedback" and 
"Praise" are least observed in fi/th grade mathematics , but occur with 
similar frequency for^reading and mathematics instruction in second • 
grade and for reading instruction at fifth grade. "Negative Feedback" 
and "Disciplining" are relatively rare. These findings, along with the 

^relatively infrequent observations of "- Conduct" pupil events, t lead to 
tlife inference that in organized instructional periods, conduct problems 
and disciplinary action by the teacher are not common. 

Grade LeVel Comparisons of Time Allocation in Instructional^ Contexts 
'in, Reading and Mathematics * - 




The types of time allocation data used to obtain measures of central 
tendency across' all classrooms have been described previously. Briefly, 
they are: # (1) tfre classroom average of the total number of minuses 
across targe x t pupils for each of ^the* three primary contexts reported for 
them; (2) the classroom average of the total number of minutes of produc- 
tive time for target pupils \n each context; (3) the average percent of 
jnirvutej? in each context for the target pupils; (Aj't^e average percent 
# of productive time fpr the ^target pupils; (5) the percent of grogs time 
in each context (total number of minutes for all target pupils in each 
context divided by .gross^ time over all contexts); and (6) thepercent of 
-gross time target pupils ^ere productive (gross productive time in 
context dividecf by gross time in all contexts). See the Procedures 

* * L 
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section (pp. 62-66) for a^-complete explanation of the calculation* of the 
measures. . v 

WHert results are reported for two of these measures (the percent of 



productive time IfPcontext and the percept of gross produetiye^time in 

context) only those classrooms in which pupils worked £n the context 

i 

were included in the calculations. In this way, classrooms in which a 
' particular context was never observed would' not be unfairly penalized by 

appearing, to have no .productive time in that content. For these two 
measures, we have indicated the actual number of classrooms in which 
pupils were observed in each context (see Tables 29 or, 31, pp., 86 and 
pp. 88)- In . the 'remainder of the table's all of the classrooms at a ^ 
particular grade level were included in the computation of the means, 

J - ■ ' r v 

medians, and^ percent of time in contexts. ? 

In general, time allocation measures (in terms of both the rank 
x ■ 

order of the number of minutes in context and the percent of time pupils 
spent in each context) should be congruent with the data reporting 
average prorated incidences of 'contexts associated wittj the pupil events. 
The time allocation data provide more precise measures of trhe ways in 
which the teachers organize their instruction, and the effectiveness and ^ 
s ' productivity of this instructional time for the target pupils. Foi; 

Example, we md*ght expect* that pupils would be less likely to be engaged / 
in productive listening when the teacher was explaining something to the 

* r • i * 

entire class and the context was Teacher-Class. On the other hand, %hep 
teachers were instructing a group of pupils, and * the context was Teacher- 
Group, the productive time might be greater because the teacher could 
more easily observe the flagging attention of one of the group members 
and intervene appropriately. These time measures provide excellent t 

\ 



o 
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. Table 24 

Average Prorated Incidence^of Teacher Responses' Observed 
.During Reading in BTES Classrooms 





Responses 


r— * 


_ Grade 2 N=41 
X * Median S,D. 


, _ Grade 5 *N=53 
X Median S.tf. 



Instructing ; 




4.04 


2,90 


4.36- 


3.11 


lt65 


4.85 


disciplining 




( 1.86 


.20 


3.35 


.83. 


.09 


1.71 


Teacher Help 




2.32 


2.33 


J 2.07 


..." 1.78 


.66 


2. .49 


'Ignoring- 




■ 2.04' 


.35 


3.30 


1.77 


.18 - 


3.03 


Moves On 




1.26 


.24 


2.14 

* 


.97 


..09 


1.67 


Negative Feedback 


• 


.47 


<10 


1.34 


\?2 


;03 


1.33 


None 


* 


62.12 


63.78 ' 


17.35 


» 66il>3 


68.38 


19.00 


Positive* Feedback 




7.05 


6.7,0 


5.93 


6.0ft 


4.28 


. 7.02 


Praise 




3.15 


1.5S 


3.81 


- • 2.14\ 


' .58 


3.36 


Questioning 




2.46 


1.73 


3.35 . 


1.04 \ 


V .11 • 


1.96 


Recognizing - 




1.66 


.83 


2.00 


. 2*. 80'- 


\I.65 


-3.89 


Redirecting 




3.47 


3.25 


3.27 


3.08 




3.79 



r 
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* - Table 25 . ^ 

Average Prorated Incidence of Teacher Responses Observed 
- During Mathematics in BTES Classrooms 



Responses 



Grade 2 N=41 
Median S.D. 




Grade 5 N=53 ^ 
' Median- SJD. 



Instructing 
Disciplining " 
.Teacher Help 
Ignoring 
Moves On 

Negative Feedback' 
None 

Positive Feedback 
Praise ' 
Questioning 
Recognizing 
Redirecting 



3.86 


2.09 


4.8T 


3.26 


2.08 


3.95 


1.42 


.29. 


' 2<48 


1.14 


.18 


2.13 


3.34 


2.^8 


/ 3.70_ 


2.54 


.60 


3.59 


2.42 




3.57 1 


2.79 


1.33 


4.17 


.34 


Ai 


y .79 


.45 


.08 


1.16 

• 


.2B/ > 


.08 


.85 


.40 


- .08 


1.03 


65^8 


66.35 


16.56 


67.90 


73.45 


21.28- 


/6-20. 


4.00 


7.62 


3.71 


1.88 


4.77 


2.35 


.48 


«.. 3&3-V 


1.22 


.26 


2.26 


1.66 


.26- 


& 5% V" 


.55 


.15 


1.25 


2.41 


.31 


3.78 


2.94r 


1.80 


•5.13 


3.34 


2.58 


. 3.85 


1.91" 


■ 1.05 


2.42 




, additional data from' which to describe organizational factors in instruc- 
don by curriculum area and grade lever. v 

\ grade. level comparisons for the, six time allocation measur es- in 
reading * As Tables 26 and 27 •indicate, more minutes are spent- in small 
group instruction at second' grade than at fifth. Teachers of other 



ad u lts s p en d ^or G-'^^e-^rrrtT individual pupiIs^Htr-WcxmC^|rade than at 
fJLfih-and tthe younger- children also spend more time' on assignments on- 
which they work independently of the teacher and of the- rest of the 
class (I-Jmjividual) . On the average, second graders spepd approxi- 
mately .72 percent (Tablj* 28) of their , reading time in'some type of group 
instructional ^activity (T-Group, I-Group 'find* A-Group) . Fifth 'graders on 
the other hand are in groue-Jjas^i^ion during reading approximately 54 

of the time v . While fifth graders were observed to fre more 
productive on the average than second graders (Table 29) in general - 
class instructional activities (T-CJass and I-Class), fifth graders were 
as productive ip, the group instructional contexts as they were in instruc- 
tion with tlTe entire, class p articipating. The ayerage percent across 
cor productjj^e^ime in the T-Class and I-Class contexts varied * 
_JLron^45^U-Class for second grade) to 63% (T-Class for fifth grade). 
The average percent of productive time in group contexts ^varied from, 53% 





U^Grou^^ at fifth graded). A most 

luctive time in the 
indlyi^TJair^context^ in reading was relatively small, Sincq a small 
amount %>f. pupil time is spent in these contetfts^ as the tables reporting 
number of minutes illustrate, we conclude that few pupils have the 
benefit of one-to-one interaction with the teacher (Teacher- Individual) 
individualized instruction (Independent-Individual). On the other 

11C 



, -> ' ■ Table 26- 

Mean Number of Minutes "Per- .Student Per 
Day. Spent in Contexts During Reading 



T-Class 

I-Class 

T-Group ) 

I-Group 

T-Individual 

I- Individual 

A-Class 

A-Group 

A-Individual 

Test 



Grade -2 . N=40 — 
X Median Range 



.27 0- 65.00 

.17 0- 25.. 00. 

17.00 15.00 0- »55-,00— 

23.08 19.00 0- 77.00 

.07 0- 4:90'. 

.54 0-i07.00 
.00 

.33 0- 29.00 

.11 0- 4.00 

.13 0- 7.00, 



6.80 
.3.63 



.30 
7.08 

.00 
-5.03 

.38 

:30 



Grade 5 N=50 
~Madd?an- — -Rasae 



7.36 l! ^9 0^-BOvOQ, 

11 ; 84 6r5tT"CPT5 ', 00 

- — t" - . * 

•11.08 ^9.50 " 0- 40.00 

19.66 16.tf0 0- 60. do^ 

.06 .02*' O-' 2.00 

' 3.30 " .25 0- 39.00 

_ ' . 00 . 00 - 

1.20' _ .29 0- 31:00 

.02 ' .01 0- . 1.00* 

4* 

- ".36 .18 0- 18 4 00 



m Table 27. 

Mean Number of ProductivewMinutes Per Student 
; Per .Day Spent in Contexts During;Reading ■ 



• 


Grade 

4 " 


2 N=40 

1 






Grade 


5 N=50 








X 


-Median 


Range : 


X 


Median 


Range 


• 

T-Class 


5.83 


.24 


0- 


64 . 00* 


6.22 


1.00 


o- 


76.00 


J. y x do o 


' £f • DO 




0- 


23.00 


ft 'XCi 




0- 


53.00 


T.-Group . x 


13.00 


12.75. 


0- 


30.00 


9.26 


7.00 


0- 


•40. 0Q, 


IfGroup 


16.23 


'l3\.70 


0- 


51 . 00 


14.60 


li.25 


0- 


•48. 0Q 


T- Individual 


' '.'25 


* • °7 • 


•Q- 


3.00 


. '.04 


.02 


0- 


2.00 


^Individual 


" 5.23 


• • 24- 


0- 


74.00 

i 


2.32 


.11 


0- 


27.00 


A-Cla^s' 


: .oo 


.00 






.00 


.00 






A-Group • " ' 


4. DO 


.33 ■ 


0- 


25.00 


.98 


.19 • 


0- 


26.00: 


A- Individual • 


.30 


.11 


0- 


2.00. 


.02 


.01 


0- 


1.00 


Test 


.18 


' .03 


0- 


'6.00 


.30 


.15 ^ 


■ 0- 


15.00 
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Table 28 

Mean Percent Total Minutes Per Student 
Per Day Spent 'in Contexts During Reading 



• 


Grade 2 N 


=40 


.Grade 5 1 


N=50 

ii 






X 


Median 




Range 


X 


Median 




Rang 


e 


T-Class 


11.28, 


• .54 


0- 


•100.00 


10.70 


1.81 


0- 


- 80." 


00 


I-Class 


4.93 


.33 


0- 


• 43.00 . 


26.22 


10.00 


0- 

1 


•100. 


00 


T-Group 


29.10 


26.50 


0- 


-100.00 


18.84 


18.00 


p- 


- 67. 


00 


I-Group * 


34.40 


•32.50 


0- 


- 74.00 


34.36 


35.00 


°f 


■ 95. 


00 


T-Individual 


.50 


.09 


0- 


- '7.00 


.18 


..04 


.0- 


- ,'7. 


00 


I-Individual 


8.33 


.30" 


0- 


- 83.00 


6,04' 


.75 


o- 

1 


- 63. 

V 


00. 


A-Class 


.00 


"•.■00 






.,00. 


. . Q0 • 






'1 ' 


A-Group 


8.98 


i.ii i 


0- 


- 47.0.0. . 


; 1.90 


>93. 


0- 


- 33. 


00 


^Individual . 


.8(3 


.21 


0- 


- 7.00 


,*06 


r ,03" 




- 3. 


00' 


Test 


•' .45 


.04 


0- 


- il-oo . 


-1.00 


.51- 


o- so^q6 j 
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Table 29 

Mean Percent Productive Minutes Per Student 
Per Day\Spent in Contexts During Reading 



T-Class 

I-Class 

T-Group 

I-Group 

T- Individual 

I-Individual 

A-'Class 

A-Group? 

A-Individual 

Test 



Grade 2'- N=40 

, _ ' Actual 

X Median Range N 



53.15 48. 25 8- 97.00 

45.10 38.00 1- 95.00 

56.36 58.50 4-100.00 

53.40 57.75 2- 89<t)0 

16.80 16.75 8- 25.00 

31.50 24.00 5- 77.00 




Grade 5 N=50 

_ • Actual 

X/' Median Range N 



33.53 31; 25 7- 85 
14.88 15.50 4- 25 
10.00 8. 50 2- 24 



00 
00 

00r 



13 
lp. 
S6 
35 

5 
12 

0 
15 

8 

3 



^-2.95 61.50 9-100.00 20 

51.39 46.50 5-100.00 28 

55.62 53.50 9-100.00 34 

62.56 70.25 17- 96.00 34 

JL4.00 14.00 -.3- 25.00 2 

3.6.22 28.75 7- 82.00 9 

0 

40.00 40.50 28- 50. 00, 3 

8.00- 8.00 1 

44.00 44.00 1 
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Table 30 



/ 



Mean Percent Gross Time Per Student Ber Day 
Spent in Contexts During Reacting 



1 — ; 1 — 

5raj3e 5 N=50 

-4 

K Median 



Grade 2 W*40 

X Median Range 



Range 



T-Class 
I-Class 
I -Group 



10.52 
5.55 



.27 0-100.00 
.17 0- 44.00 



27.35 25.50 0-100.00 



I-Group 


36.20 


35. Op 


0- 


74.00 


L-lndividual 




•OP 


0- 


7.00 


I-Individual 


9.08 


\ Jo 


0- 


•89 . 00 


A-Class 


.00 


I .Jo 






A-Group 


9.30 


.47' 


0- 


49.00 


A-Individu^l 


.93 


.32 


0- 


7.00 


Test 


.63 


• ?1 


0- 


17.00 



? 



10.72 2.17 0- 

I '25.66 12.50 0- 

ly.oo 20.00 o- 

i 35.44 36.50 0- 

.16 .02 0- 



6.14 
- .'00 

1.98 
.0.4- 
.88 



.63 0- 
.00 

•96.0- 
.02 '6- 
.45 0- 



80.00 
100.00 
67.00 
95. 00 
7.00 
67.00 

36.00 
2.00 
44.00 



V 
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Table 31 

Meian Percent Gross Productive Time Per Student 
Per/Day Spent in- Context During Reading 




Gr^de 2 N=40 

Actual 

X Median ' Range N 



90.75 72-100.00 13 

86.00 3-100.00 10 

89.00 9-100.00 36 

76.25 12-100.00 35 

93.75 50-100.00 5 

80.00 50-100.00 12 

0 

81.00 41- 98.00 15 

95.83 55-100.00 8 

41.25 12- 87.00 3 



Grade 5 N=50 



Actual* 



X Median Range 'yf. 



86.40 90;17 63-100.00 if 
69.46 . 75.50 f 10-100.00 28 
85.62 90.17 35-l#0.00 34 
75.21 76.50 50- 95.00 34 
75.00 75.00 50-100.00 2 
82.67 87. 00/' 6^100. 00 9 

0 
3 
1 
1 



79.67 ..80.50 71- 85.00 
71.00 71.00 • 
86.00 86.00 
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hand we would expect that when time was allocated to individual or 
independent assignments, the time would be more productive since the 
pupils Would be receiving either maximum attention from the teacher or 
using materials of special interest. The data on productive r-satling 
time in these contexts does not support such assumptions when*, productive 
time is averaged acro.ss pupils. When the gross percent of productive 
time is computed (table 31), the values vary from 78% to 83% productive 
in the Independent-Individual contexts. We suspect that the range of J 
percents of productivity reflect the fact that the Independent-Indivi- 
dual assignments may be a type of classroom management some teachers use 

with pupils who cannot otherwise work in a group or on a common class- 
t, * - . — 

room assignment. Those classrooms wher^the proportion of productive 

time in this context was high were presumably those fohere individually- 
planned, individualized instruction was provided to pupils who 
« 

could sustain their attention to the task for long periods of time 
without frequent supervision from their teacher. 

Grade level comparisons for the six time allocation variables in 
mathematics . Teacher instruction 9f the entire class is the context in 
which most time was spent in second grade mathematics (Table 32). In 
fifth grade, equal amounts of time were ailptted to contexts in^hich 
the teacher was directing the work of the entire class or in which 
pupils were working independently on a ^common assignment. The amount of 
time spent in Teacher-Group contexts was similar at both grade levels, 
so that the primary difference in time allocation in mathematics between 
grades is in the relatively greater emphasis on the Teacher-Class con- 
texts at second grade (Tables 3A and* 36). The average proportion of " 
• * 

pupil productive time in context (Table 35) is higher in Teacher-Class 
instruction t&an in any of the other contexts. However^, when productive 



Table 32 

\ 

Mean Mumber of Minutds Per Student Per Day 
Spent in Contexts During Mathematics 



/ 


Grade 


2- 'tf=38 




Grade 


5 N=46 


- 


\ 


A 


Median 


Range 


X 


Median 


Range 


T-ulass 


14.66 


12.00 


0-50.00 


13.49 


10. c 06 


0-50.00 


I-Class 


5.92 


■ .41 


0-.28.O0 


14.60 


13.25 


0-58.00 


T-Group 


5.79 


.83. 


0-33.00' 


5.86 


.44 


0-27.00 


I-Group 


7.90 


4.50 


0-34.00 


11.11 


6.50 


0-52.00' 


i- Individual 


.00 


.00 




.02 


.01 


0- 1.00 


1- Individual 


1.90 


.11 


0-30.00' • . ' 


, .-.b9 


4° 


0-17.00 


A-Class 


.00 


.00 




1.36 


.49 


0-40.00 


A-Group 


.92 


.27 


0-10. 0(X 


.04 


.02 


0- 1.00 


A- Individual 


.18 


.06 


o- 3.00 


.04 


.02 


o- i.oo 


Test 


.24 


' .12 


0- 9.00 


.78 


.•09 


0- 8.00 



^Table 3*3 



Mean Number of Productive Minutes Per Student 
'Eer Day Spent in Contexts During Mathematics. 




At Individual 
Test: , • 



Grade 2' N=38 
X Median 




.Grade 5 N=46 
.X "V; Median .'Range 



LI. 55- io;oo 
U.ll ,36 



4.82 
5.84^ 

.00 
1.53. 

.00", 

.08 
.24 



•.83 
3.5Q. 
.00 

.06 

4 9 

.04 
.12 



0-41 . 00^ 
0-24100 
0-29.00 
0-27.00 

0-24.00 

v 

0-10*. 0Q 

o- r.oo 

0- 9.00 



10.42 

10.09 
\ ' 
4.44 

8.42 

•* .00 

^36 
^LOO 

.0 



.•69 



7.67 
9:75 

.37 
5.63 

.00 

.07 
x .33 1 
^02. 

^09 . 



[ 0^33, qo 
•0-40.00 
0^2? :0O 
0-45/00 

■s«" 

'0-13.00. 
'Orf31,-00 
Qr 1.00 

- d- l.ob 
<.ov s'.oo 



Table 34 > 

Mean Percent Total Minutes Per Student Per Day 
#r * Spent in Contexts During Mathematics 





Grade 


2 N=38 




Grade 


5«, N"=46 










-^Median 


Range 




Median 




'Range 


• 

1-CIass 


40.42 


33.25 


0-100.00 


97 


\ 

o o 

IJ J ( 


21.25 


0- 


100. 00 


-L-Class, 


.13.53 


.81 


0- 68.00 . 


31. 


09 


31.75 


-0- 


100.00 


T-Group • 


15.11" 


• 2.75- 


0-100.00 


11. 


98 


> '. 88 


0- 


59. 00 


I-Oroup 


21.74 


11.50 


0- 74.00 


22. 


13 


14.75 


0- 


" 82.00 


T-Iqtiividual 


.29 


.03 


p- 10.00 




02 

1 r 


.01 


0- 


. 1.00 


r-tmi^\d.dual 


3.92 


.34 


0- 48". 00 ■' 


' 2. 


09 


' .20 


0- 


.50.00 


* ** 
A-^lass 


.0*0 


> . .00 






49, 


.91 


0- 


73.00 


A-Group 


2.66 


' ' .5*3 


0- 34.00 




07 


.02 


0- 


2.00 


A-Ind ividu&l 


.45. 


.08 


0- * 5.00 ' 




11 


.02 


0- 


4.00 


Test 


.79' 


. .41 . 


6- 30.00 

l 


1 2. 


16 


.18 


0- 


40.00 

4 
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Table 35 



Jut 



Mean Percent Productive Minutes Per Student^ 
Per Day Spent in Contexts During Mathematics 




// 




Grade 
X 



Median 



Actual 
Range N 



21.83 
9.75 

42,00. 



68.75 15-95.00 
45.50 6-95 

88.00 
7-87". 00- 



4^40700- 
8-50.00 





16-.-0OL. 
9.75 
42.00 



16; 

19- 

.22' 
1 
7 
0 
6 

*4 




Grade 5 



Median 



Actual?- 
Range N • \ 



67.07- 
58.71 
46.75 
47.89 
"3.50 
17.67 
62.50 
13.00 
6.00 
30.43 



70.7,5 13 100.00 29 

57.00 lV 97.00 28 

44.50 3- 93*00 20 

48.00 .8- 87.00 28 

3.50 1- 6.00 2 

16.50 1- 39.00 3' 

0 62.50 47- 78.00 2 

13.00 4- 22.00 2 

6.00 4- 8.00 2 

28.00 11- 50.00 7 




Mean Percent Gross Time'Per Student Per Day 
bp&nt In Contexts During Mathematics 




v 
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Table 37 

Mean- Percent Gross Productive Time Per Student 
Per Day Spent -In Contexts During/Mathematics 



95 



T-Class 

I-Class 

T-Group 

I-GrdUp 

T-Individual 

I-Individual 

A-Class 

A-Group 

A-Individual 

Test 




Actual 
ange N 



Grade 
X 



Median 



19.2b 81.83 55- 99.00* 27 
73.06 71.50 33-100.00 
83.79. 92.75 36 . 10£<l50 19 
74.55 79.50 30^0.00 22 

1 



Actual 
Range , N ^ 



100.00 100.00// 
81.57" 80*50 67-100.00 



92.50 95.00 75-100*00 
85.25 92.00 57-lOOyOO 
100.00 100.00 t 



7 
0 
i> 
4 

i 




52-100.00 29 
30-100.00 28 
50-100.00-* 20 
36-100.00 28 
20-100'.00 ' 2 
M*-78.00 " 3 
•68- 78.00 2 

70- 75.00 2 
76-100. 00' 7 



/ 
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time in- contexts was computed using gross t,ime '(Table 37), the Teacher- 
GrQup mathematics contexts were more productive. * Pupils were in Independ 
ent-Individual contexts and contexts where an adult other than the 
teacher was in char'ge of instruction in only eleven second g*ade class- 

* 

rooms'." While the average productive pupil time in these context&'was 
lower than in other contexts, the percent of productive gross time was 
high. If the observer judgments o.f' productivity are accurate, time " 
Spent in Independent— Individual, Teacher- Individual, Adult-Individual " 
and Adult-Group contexts on the average is not generally productive time 
in mathematics. However, when gtoss time is considered, the productivity 
rate for the classroom overall' is as high- In the individual context as 
in other settings. Such a finding suggests thajt otily the small number * 
of pupils who can tolerate rather long' periods of time working indepen- 
dently are the ones who ai^e judged to be doing productive work. 

Partial Correlations of Observation Variables With Reading Achievement 

% *■ ' 

Outcomes . * » v ■ * 



'?tve .outcome measures 'defined the criteria which were available* to' 
evaluate the validly of the APPLE observation -variables. One of tfiese 

' * ' ' - X * ' ■ ■ . 

Measures was a nationally'^andardized achievement test, the California 

Achievement Test in Reading: Comprehension. "Three v other measures, of *•* 

reading achievement were developed as part of the Phase IT BTES Pupil* 

• - * ' ' ' » a : 

Test Battery. They were tests in reading achievement , reading applica* 
* « -> 

tion, and^ decoding. The fifth Measure was an' overall score combining 

the store from the Calif ornia Achievement Test and the *three project 



developed measures., AddiUrtfCal information about these measures and 
their psychometric properties is presented in Volumes It and IV. 

We used a partial correlation of the observation variables with 
the end" of year measure controlling for the fall test scored as the 
validity statistic. In common with most other observation Systems' 

- 124 




• ' ' * " . . 97 i 

APPLE produces variables which are typically assymetrical and contain 

relatively few class intervals ♦ In % such .situations the correlation 

coefficient is. limi'ted in. the values it 4 can~as*sume. However, since 

no* other' analytical method seemed as suited to our needs, we tiopfcd 

' • *. * . * 

to insure that all potentially valid measures be included for further 

• •< _ * « 

study by employing a less conservative probability level for ascer- 
taining statistical significance than is conventional. Accordingly, 
we accepte /a probability level of .10 t>r less that the partial 
correlations .differed significantly from zero.* ; 

Partial correlations of prorated incidences' of pupil events with 
reading outcomes * Several pupil 'characteristics were significantly* 
related to achievement outcomes (Table 38) -j The greater the frequency 

y * ' * * » ' 

' * 4 v * * 

of talking out inappropriately in class (V- Talking") the poore.©^t"he 

average achievement of that class in four of the five achievement \ 

** ' • " ■ 

measures in grade .two. At' She .second grade level, observations of pupil 

' • ' * ' 

behavior .which wejfc"-|ieutral with respect to "Engagement 11 a^d "Conduct" 

• * \ 

were negatively correlated to many outcome measures. These results 
an' inverse relationship between the end-of-year achievement 
outcomes and the observed frequency of pupil behavior 'whicfi «/as perhaps 



not, consistent with teacher ideas of ideal behavior, but not*. necessarily 
serious enough to , warrant teacher 'intervention. * 

At second grade, more frequent evidence of positive achievement in 

v 

phonics ("+ Phonic Skills") was positively correlated with reading 
achievement over two achievement tests, while the observation of achieve- 

ment in phoniqs was negatively correlated with outcomes at fifth grade. 

- f 

j^. cati offer the tentative interpretation that 7 the more' pupils are 
observed to* be working on\,phonics activities in second grade, the better 
f the chancres that their scores in reading application and decoding will 
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improve significantly at the-end of the .year. However,, observations of 
pupil activities in phonics at fifth grade suggest pupils who do, not 
have word attack skills are those who are experiencing reading dif fi-^ 
culties. 

"Oral Reading", while a common occurrence at second grade, is not 
significantly related to 1 end of year second grade achievement. In fifth 
grade, where opportunities for oral reading are not as common, the 
frequency of observed oral reading of the target pupils was significantly 
and positively related to reading achievement. 

Of some interest, 'because it violates conventional wisdom, is the 

finding that "+ Participation*/ was negatively related to application and* 

* • * 

decoding outcomes iti second grade and .decoding and total reading, in ( 

fifth grade. In reading, "+ Participation" was most commonly used to^ . • _ 

* X * ■ 

label those observations indicating that the pupil'was joining in 
appropriately as a member of "a tearcher'led reading group, *It then 
appears* that teacher' led groups as arclaj^robm organizational strategy 
may not be the most productive* "of pu^pil 'growth in the reading* skills 
tested by the application' and decoding tests. The results of the sub- • 
sequent^n^ys?s~address this issue more 'directly. . * 

Partial correlations of prorated incidences of . instructional contexts 
with reading outcomes* Several context varlabie^^I^ble 39) were signi- 
f icantly related -to reading achievement outcomes.-, However, only a^^bsiei 
description of the natufe and direction of the relationships will be t . . 

• ^ * 

offered here. It. is i^por tant^to note that the use of th^se context 



variables insolation o'f, teacher or .pupil 'behavior obscures the potential 



retatipnship fcetween management and instructional skills of teachers ahd 



their impact on'behavior and lear^ing;^ Especially , as cfompared to 
the resul^^fot^tKe^context time allocation variables, the context 

" 127 • . 



100 



en. 



eO 
H 





50 
*j 










0) 








03 






in 


<y 


-o 


±-> 


<o 


c 


0) 


o 




o 




rH 


1 


cO 


CO 


C 


CD 


o 




•H 


o 


i-J 


CJ 


a 


C/3 




u 




o u 


E 
O 


G 


o 


>H 


4J 




3 




O 


CO 




or 


rH 


rH 


rH 




• CO 




tL. 


•H 




V-i . 


CO 


O 






W 


'00 


PL 


r- 
•H 


5! 


i—l 




«H 


•S 


ro 








c 


« 


0 


i 


u 




o 


CO 




0) 


CO 

c 


or 


o 


cj 


•H 
4J 


CO 


CO 


cu 




a. 

o 




o 


M 


u 


O 




O 


JO 


' % 


00 






•H 


5 


JJ 


u 


M 




'CO 


CO 


(X 



Cv 00 

rH S *H 

co cy TJ 

U "T3 <0 

O CO 0) 
< 



c 

c fi 

> 

?3 CD 

a) h 



oo 
c 



O ±J 

cj O 
CJ 
Q 



C 

o 

00 "H 

C u 

co 

t? a 

CO -h 

0) rH 

a. 
< 



OO C 

c o 

•H *H 

"O ,co 

CO C 

0) 0) 

cs: x: 



■6 



c4 



CM 



O 



ao 
O 



O 
O 



CM 

I 



o 
o 



* 
o 

CM 



ro 

ro 



o 
O 



O 
O 



uo 
O 



CO 



o 



rH 

r 



o 



O . *fO 

CM ^-O 



CM 

o 



ro 
O 



o 



O 



m 
o 



o 



oo 

rH 



* 

CM 



O 



.CM 

o 



ro 
O 



o 



ON 

o 



00 

o 



St 

o 



* 

ro 



I 

* 

CO 



ON 

o 



o 



m 
o 



C 



o 



CM 
CM 









rH 










idua 






CO 


Cu 


. > 






0) 




•H 






CO 


o 


TJ 






rH 




C 






O 


o 


H 


CO 




1 


1 


, 1 
«a u 


CO 


Cu 


4-J 


4-J 


-co 


3- 


c 


c 




rH 




•or 


■ 0) 








•a 






7 


e> 


c 


c 


§ 


er 


1 


0) 


. el) 




u 


Pu 








n 


Ho 


de 


a) * 


ac 




C 




• c 


0) 


3 


rH 






H 



•a 

M 
O 

i 

■ u 

a) 

H 



, i 



v ' vl 



EMC : 



126 



101 

incidence variables represent a much "grosser' 1 cut of the .available 
data. Since a careful analysis of the relationship between the time in 
context measures and achievement will be reported later in this section, 
the results to be discussed here are abbreviated in comparison to those 
which will follow. 

Thus, we found that "Teacher-Group" and "Independent-Individual" 

0 

contexts correlate significantly with the California Achievement Test in 
grade two. "Independent-Group" and "Independent-Class" contexts are 
positively correlated at a significant level with the decoding and 
reading achievement measures respectively in grade fiv^C On the other 
hand, the frequency of the "Independent-Group" context at second grade 
is negatively related to the reading achievement measure. 

Partial correlations of prorated incidences of teacher activities 
with reading outcomes . It is of considerable interest that teacher 
activities which involve verbal behavior in the organization and intro- 
duction of lesso/is are negatively correlated with achievement outcomes 
(Table 40). "Academic Organizing", "Asking" questions, "Discussion",— 

and "Instruction Giving" are all significantly and negatively correlated 

i 

with the Reading Comprehension Test of the California Achievement 1 Tests- 
in second grade. "Academic Organizing" and "Instruction Giving" are 
inversely related to total reading achievement also at the, second grade 
leyel. "Supervising" pflpils, a type 'of non-interactive activity, is 

4 t 

also negatively correlated at a significant level with three of the five 

I 

achievement measures. The only teaching activity with uniformly posi- 
tive correlations with reading outcomes is "Checking" . From these 



results we can, of fer the tentative hypothesis- that the 



more the teacher 



is involved in organizing the class for instruction, and the less the 
teacher is involved in ^checking to see if pupils are under standing their 

129 • I 



work, the poorer the end of year reading. This hypothesis is an inter- 
esting one since -it is the .verbal behavior of teachers which is often 
the object-of supervisors evaluation of classroom functioning. 

Partial correlations of .prorated incidences of teacher responses 
with reading outcomes . While the incidence of teacher "instruction 
Giving 1 ' activities is negatively correlated with outcome (Table 40), the 
incidence of teachers 1 responses to pupil behavior by "instructing 11 is* 
positively correlated with second grade reading (Table 41). This Inter- 
vention can be understood as a npnjudgmental response in which v the 
teather repeats the task or shows the pupil how to do something. While 
"Redirecting" a pupil whose beha^or is perceived to be inappropriate is 

also a rather neutral teacher response, the, greater K the frequency of 

* *■ ' 

these responds the less likely the achievement will b£ as high in 

» , ! ■ 

spring as would .have been predicted by §he fall scores*. We^inf er" that 

classrooms in which trhere is -more need for teacher ..intervention of this • 

type ar^ those' in which pupils are working less productively . Evidence 

of "Positive Feedback" and "PraisV to pupils is' positively correlated 

with second grade reading performance. "Positive feedback", on the 

other' hand, was negatively correlated with the ETS reading achievement 

test at fifth grade. The response 'Xoves On", used when the teacher 

failed to acknowledge a pupil behavior, is negatively correlated with 

achievement at 'both grade levels. TheJ'Moves On" response is one in 

which the teacher responds to pupil behavior by calling, on another pupil 

or moving on to another part of the lesson. The more this occurs 

during reading lessons, the more likely it is that reading achievement 

at the end of the year will be lower than expected. 
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Again it sfribuld be noted that considering the relationship of 
teacher responses to pupil outxoi^fes in isolation of their interaction 
with instruct ionalT^cont^xt^ teaching activities, and pupil behavior 
gives an incomplete picture of the "true" nature of the relationships 

- *; 

among the variables. Perhaps some of the reversals in direction of 
the relationships across grade level for certain responses (e.g. -"Posi- 
tive Feedback") are a consequence of differing patterns of instructional 
contexts and teaching activities across grade levels. 

✓ . - * 

Partfial Correlations of Observation Variables with Mathematics 

Achievement Outcomes 

. Two nationally standardized mathematics achievements tests (the 

Math" Cone eptf and Math Computation tests of the California Achievement ' 

ests) k and one newly developed one, the Mathematics Applications Test, 



.were the criteria for evaluating mathematics outcomes. In addition, a 
total score derived from the combination of measures for the three tests 
was computed. These tests are described in Volumes II an4 IV. 

Partial correlations of prorated incidences of pupiljevents with 
mathematics outcomes . In second gr'ade classrooms where there is a high 
frequency of inappropriate conduct, obvious inattention to the classroom 
work, and talking out of turn, there is more likelihood that the achieve- 
ment scores in mathematics will be less than predicted at the end of the 
year (Table 42) . , The frequency of ^observed pupil involvement yith 
„ lessons implies by the lf + Engagement" events in secohd grade is signifi- 
cantly "and positively related to second grade mathematics achievement on 
two measures. " 

Observable l^havior associated with number concepts was not relate" 
to. achievement except in^the case of " Number Concepts" in fifth griade 



i 

- *. Table 42 . ; 

, 'Partial Correlations of BTES" APPLE Variable (Pupil Events) with 
' • t Spring .Qutc&me Scores' Controlling for Fall Scores - 

. Mathematics Tests 



u , Total 
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and "0 Number Concepts 11 in second grade. However, the pattern of the 
correlations having to do with "Number Concepts" were neither uniformly 
positive nor negative. A careful review of the, test items and the 
characteristics of the classroom observation records might clarify thesfe 
apparently, contradictory findings. . * 

Evidence of inattention which is not disruptive of classroom work 
("0 Inattention") is negatively correlated at a significant level with 
all of the fifth grade mathematics achievement measures. .This finding' 
can.be interpreted to mean that when there is a great frequency of 
observations of possible inattention to the" mathematics lessons at fifth 
grade, there is t quite likely to be, poorer end-of-year achievement in 
that classroom. 

Partial correlations of prorated incidences of instructional 
contexts with mathematics outcomes . In general, the frequency of 
"Independent-Group" and "Independent-Class" contexts was negatively 
related^ to achievement outcomes. While a clearer picture of the quality 
of pupil behavior in these contexts in mathematics will be provided when 
we present t&e correlations of the time alldcation, data with achieve- 
ment, the. results on Table 43 suggest that classrooms in which pupils 
are working independently, of the teacher in mg^Jiematics most of the time 
are those which are more likely to have poorer achievement outcomes. 

Partial correlations of prorated incidences of teacher activities 
with mathematics, outcomes' . Several teaching activity variables are 
significantly correlated with the mathematics achievement measures 
(Table 44). However, only one of them, "Asking," has a significants 
relationship with more than one measure. Instructional activitiy in ~ 
mathematics is not very observable, except for demonstrations at the 
board. Pupils are working independently on workbook or textbook aS0ign- 

135_ ' ; - ' . 
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Table 43 



Partial Correlations of BT&S APPLE VariablesYln'structional Contexts) 
with Spring Outcome Scores .Controlling tor Pall Scores* - 

'Mathematics Tests | . - -° .*/ 



— ■ - / ^ v 

Math Math • • • Math r 
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-.16. 

-.04 

•»23* 



.15 „ *-.31* 
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Partial Correlations of BTES APPLE Variables (Teacher Activities) with 
Spring Outcome Scores Controlling for Fall Scores - Mathematics .Tests 

















. - Total 
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ments, and teachers circulate around the room checking work, helping 
those who need it, or remaining at their desks available to those class 
members who need assistance. (Refer to Table 14 for the raw frequencies 
of teacher activities.) The prorated frequency of the "Asking" teacher 
activity was negatively correlated with outcomes. Reference to the 
.lexicon definition presented earlier (Table 15) generates the hypothesis 
that teachers who spend time asking questions of,, their second grade 
pup32s are those whose lessons are not understood; possibly their assign- 
ments are at an inappropriate level of ^difficulty 'for the pupils. 

Partial correlations of prorated incidences of teacher responses 
with mathematics outcomes . Even though teacher activities in ma them— - 
a tics do not provide a clear picture of teacher differences which are 

a 

related to outcomes, an examination of the iiumber of significant partial 
correlations of teacher responses with achievement does show some 
interesting findings (Table 45). Teachers who are observed to employ"' 
more management techniques at second grade^ such as "Disciplining" land 
"Recognizing", are those whose pupils are more likely to have poorer 
achievement. An explanation of these findings might be that the dif- 1 I 
ficulty level or the interest level of the mathematics assignments is 
possibly not appropriate for some pupils and in those classes^the teachers 
must intervene more often to keep the class in order. These may also be 
classrooms in which there are Unrealistic expectations that pupils work 
by themselves and be attentive to the task. 

, At the fifth grade level, "Teacher Help", to pupils and providing 
"Positive Feedback" and "Praise" are teacher response Variables which 
are positively correlated with ^end of the year mathematics achievement. 
All but one of the teacher response variables have a significant rela- 
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Table. 45 

Partial Correlations' cff BTES APPLE Variables (Teacher Responses) with 
Spring Outcome Scores Controlling for Fall Scores - Mathematics Tests 









Total 


CAT Math 


{ * CAT Math 


Math 


Academic 


r . ? Concepts 


Computation 


Application. 


Math 


2 5 


2 5 


2 5 


2 5 



Instructing - -.14 .13 .07 .03 -.03 -.17 .03 -.02 

-D isciplining -2-1* ,09 -.06 - _J)4 -.16 _JLI ^21* .H3_ 

Teacher Help -.07 .1-2- ' -.-16^ .34* .07 • .31* -.15 .28* 

Ignoring .03 .10 -.02' -.06 .05' .27* .05 .09 

Moves On -.01 .-.03 -.08 -.22* .05 .15 .05 -.07 

Negative Feedback - -.08 .03 .12 -.08 -.27* .26* -.00 ' .11 

, None .06l.- -v» - " ' 768 -.09 .07 -.23* -.00 -.20* 

Positive Feedback -.03 .23* -.05 .01. -.12 .38* .04 .35* 

Praise' -16 .29* '.04 .04 .05 " .11 .12 ,,--<25* 

Questioning .17 .08 .08 .06 .24* .18 .08 .14 

Recognising-. '' -.32* -.28* -.01 -.28* -.28* -.31* -.08 -.41* 

- Redirecting ' .25*. .01 '-705 " T22* -.28* -.21* .02 .09 



* p<. .10 
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tionship to one or more measures of mathematics achievement. These 
, findings sug'gest that the more the teacher interacts with students about 
their mathematic lessons the better the learning outcomes. 

-„ ^ 

Partial Correlations of Observation Variables with Pupil Attitude Outcomes 

Four attitude measures, were available from the fall and spring 
testing periods. These are described in other volumes of the BTES final 
report, so we will comment only on the methocf used to obtain the outcome < 
measures. Each second and fifth grade pupil took a reading and mathe- 
matics attitude test and a reading and mathematics survey. The attitude 



ERLC 



measure asked the pupil to indicate wh£the£ he had positive, neutral or 
negative feelings about reading and ^mathematics activities. The survey 

asked for a variety of information about self concept with respect to, 

< 

peers, school, and inter ests^-A^jiif f erent types of activities. Since 
the reading and mathematic^sectiotts^-of^^ach pair of instruments corre- 
lated highly with one another^ there was no reason to keep them separate 
as outcome measures. We decided, therefore, to use the total scores 
both from the reading and' mathematics suivey and from the reading and 
mathematics attitude measure as the two outcome measures in the partial 
correlation studies of our observation data. 

Partial correlations of prorated incidences of pupil events in reading 
and in mathematics with attitude outcomes . At. second grade ,f + Engagement" 
events in reading instruction correlate positively with the total attitude 
measures (Table 46). Evidence of anappropriate behavior in the classroom, 
as reflected in the average incidence- of Conduct", and 11 Talking" 
events in reading, correlate negatively with the total attitude measure. 
We could hypothesize, tentatively^ that with respect to attitudes toward 
reading and mathematics, second grade reading lessons where target 
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pupils are observed to be positively involved in the activities of the 
class, and where they are not misbehaving or talking inappropriately are 
those classroom^, with the most positive changes- in attitude* at the end 
of the year. C&servations of " Conduct 1 ' events in fifth grade reading 
aire also negatively correlated with the attitude outcome. Evidence of 
opportunities for "+ Oral Reading 11 are positively correlated at both ' 
grade levels. We could infer generally that at both second and fifth 
grade, classrooms 1q which there was positive evidence of involvement in 
1 the classroom reading activity and an absence of inappropriate behavior 
would be thos^class rooms where attitudes toward reading and mathematics 



would improve over the year. 

> ; * ■ *> f . ; f * 

When we review p\ffii^6Ven^<±n^mathematics at the second grade 

level, it is of interest to? note that* incidence of M + Academic . ' 

* • S 
Performance 11 and Attentiveness" events are negatively related to the 

total attitude measure. A good hypothesis to test would be that events 

such as "+ Attentiveness" occur in classrooms where the pupils are 

expected to be listening to the teacher explain a mathematics lesson — 

for example, teachers might be .asking questions of pupils to see if they 

know answers to problems. t In these situations ,^isjis possible that the 

greater the incidence of these patterns* of activities, the less the 

opportunity for the pupil to explore and extend his mathematics skill at 

his own rate of learning. Consequently, attitudes toward reading and 

mathematics become less favorable. 

" At fifth grade, evidence of ,! + Participation 11 and "+ Volunteering 11 ' 

in mathematics lessons are positively correlated with the attitude 

measure, suggesting that attitudes improve in classes where pupils are 



actively involved in interaction with the mathematics lesson material. 



14 



2 
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Evidence of some inappropriate, or marginally acceptable behaviors in 
fifth grade mathematics lessons is negatively correlated with attitude 
outcome. 

Since several of the pupil atftvey items ask questions about how a 

pupil ^judges his abilities with respect to those of his peers/ it is not 

surprising that some of the correlations between *piipil event variables 

•# 

evidencing appropriate pupil behavior in reading and mathematics and the 
survey measure are negative at fifth grade. ;A pupil might be observed 
to be productively participating in classroom activities, but might view 



his^p.wtV4>erf prmahce as. less adequate than, that of some of his peers. In 
such instances, one would expect a negative relationship to the survey 
i£ems. However, at second grade, M + Academic Performance", "+ Attentive- 
ness", Direction Following 11 and "+ Participation" during reading 
activities have a significant positive relationship to the survey measure. 
Since quite *a different pattejn emerged at' fifth grade,* this suggests 
an age difference in the implications of these types qf observed 
positive pupil behavior on attitudes toward self worth. 




Partial correlations of proratec^incidences of instructional contexts 

in reading and 4n mathematics with attitude outcomes . Examination of 

the partial correlations between incidences of contexts with the 

attitude and survey measures is not by itself too productive of 
f * 

immediately^estable hypotheses (Table 47). In all. but one case, those 
contexts with significant relationships were those in which the teacher 
was in' charge of instruction, either the "Teacher-Group" or ' "Teacher- 
Class" contexts. Grade level differences in the relationship also 
appear. "Teacher-Class" instruction in both reading and mathematics 
at the fifth grade level is positively ' correlated with the attitude 
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measure. At second grade7~incidence of this' context is positively 
correlated with the survey measure for mathematics and negatively 
correlated for reading lessons. The reverse is true of the relation- 
ship between incidence of th,e "Teacher-Group" context and the second 
grade survey measure. In this case, "Teacher-Group" instruction in 
reading is positively correlated with the classroom average survey 
scpre while the incidence of this context in mathematics is negatively 
related to scores on the survey. Since "Teacher-Group" instruction is 
more common in reading than in mathematics, it would be necessary to 



find out more about the kinds of teaching activities and teacher 
responses which take place in reading and mathematics groups in order 
to clarify these differing' significant relationships. These are 
hypotheses which should be tested in future researc^. 

Partial correlations of prorated incidences of teacher activities 

i 

in reading and in mathematics with attitude outcomes .^ The analysis of 

i 

the relationship of teacher activities"' in reading and irlathematics ■ to 
the attitude and survey measures also demonstrates the possibility of . 
positive and negative correlations* of some teacher activities depending 
on the type of instruction and the grade level (Table 48) . Being "At 
Board" during second grade reading instruction is positively correlated 
with the attitude measures but negati^ly correlated when it occurs in 
fifth grade reading. The incidence of ttsirtg "At Board" during second 
grade mathematics is negatively related :o both the attitude and the 



survey jftea'sures. However, the frequency 



with which teachers ar^ "At 



Board" in fifth grade mathematics- is positively correlated with the 



attitude measure. 
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Some, teacher activities which are related to attitude are always 
positively correlated with attitude and survey outcomes whenever the 
correlations are significant. Some of these activities with significant 
positive relationships are "Checking' 1 , "Listening" and "Supervising", 
The activities which always have a negative relationship to attitudes 
when significant are "Circulating" and "Instruction Giving". "Checking", 
"Listening" arid "Supervising" imply positive or neutral interest of 
the teacher, while "Circulating" and "Instructiom^i^ing" suggest 
-mev^ment-aFoundr-r-equ-ir-ements-of -the ies&OTw— Some o-f these -activities 



correlate positively with achievement because they are evidence of 
teacher's degree of involvement in the work of teaching. However, 
these same activities may relate negatively to attitude outcomes % 
because such teacher responses cause children to feel less positively 
about their abilities to do- the school work. 

Partial correlations of prorated incidences of^Tea'cher responses 
in reading and in mathematics with attitude outcomes . Teacher responses 
to pupil behavior have differing relationships to attitudes depending 

on whether the teacher behavior is observed in reading or in mathematics 

\ 

and whether the pupils are second or fifth graders. In reviewing 

* \ 

x 

Table 49, it is important to consider that pupil attitudes toward **\ 
reading and mathematics, as well as their feelings of self worth, will 
be dependent not only on how the teacher responds to them, but^aiso on 
how this re-sponse affects their self perception. For example, the 
incidence of "Disciplining" was negatively related to '.reading attitude 
when it occurred in second grade reading instruction, but positively 
associated with the survey measure when it was observed in second \ 

\ 

grade mathematics. One could explain this apparently disparate 
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finding by hypothesizing that when teachers, discipline pupils in , 
reading, pupil^are less likely to respond positively to items assessing 
the degtee to which they enjoy reading activities. On the other hand, 
pupils of the. same age may perceive tlie legitimate use of teacher 
discipline as an awareness of individual differences in th^fcr own 
behaviors, which in turn changes their se.lf concept. At the least* we 
would have to state that the incidence of disciplining in a classroom 
cannot always be judged as lowering the general attitude in the 



classroom. / 

^WhiJLe one cannot infer from these data the the ex^fcnt of the use s 
^qf disciplining will be negatively related to the average classroom 
level of attitude and self concept, neither can one infer that the 
frequency of "Negative Feedback 11 is a predictor of decrease in atti- . 
tude or self concept. While the number of pupil events which receive 
a teacher^fSsponse of "Negative Feedback" is inversely correlated with 
the second grade pupil survey results for.^Teading, instances of 




'Negat 



back" in mathematics have positive correlations with ^the 



survey measure in secon 



id the attitude measure in fifth 



x 



grade. The extent of "Positive Feedback" is always positively related 
to attitudes when the partial correlations are significant. Neutral 
teacher management techniques and teacher responses such as "Moves On" 
and "Redirecting" are also mostly positive jLn^ relation to both attitude 
outcome measures. One gets the impression from the examination of the 
data and from the' analysis of the relationship between teacher activi- 
ties and pupil attitudes, that- teacher b^havitfr\which calls attention 

c 

to children, and perhaps* places them in anXuntavorable comparison to 
their classmates, may affect attitude changes even thought these same 
teacher behaviors have a positive relationship to achievement outcomes. 
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Overview, of the Relationship of Observation Variables to Achievement 
and Attitude Outcomes ~ 

Even though we^have prepared numerous tables to present the 
findings from the APPLE observation phase of the BTES project, and 
another set of summary findings might seem unnecessary, we believed 
thpt tabltes which present an overview of the most promising observa- 
tion variables would be valuable.' Accordingly we prepared Tables 50, 
51* and 52 to summarize the relationship of pupil events, teacher 



activities and teacher responses with the outconte measures. In these 
tables we have indicated, within achievement area and grade, level, 
those variables which have a significant relationship to two or more 
outcome measures. This, of course, \s an arbitrary criterion. 
However, in the absence of more detailed information on the psychor- 
metric properties of the achievement measures, we felt that it was a ;y 
*f air criterion to use.^ The direction of the relationships is indicated 
with the use of V f and signs-. In attempting to summarize the 
findings from the attitude measures (see Tables 52A, 52B, and 52C) , 
we decided that a variable which was significantly correlated for 
both grade leyels within each outcome measure^wdiild be included. In 
these summary tables we have not included the data reporting corre- 
lations with instructional context because a more detailed^. presenta- 
tipn of the correlations between time allocation in context measures 
and outcome will follow. 

Forty-seven APPLE variables (excluding instructional context) 

*■ i - 

were idenditified from the total set of observations on the basis of 

i 

the frequency counts in^the observation record. There were 20 pupil 
events, 15 teacher activities and 12 teacher responses. Of these, the 
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V 

''great majority-aould be considered, of -potential vajjae when variables 
were /included which were significantly related to either reading or 
mathematics achievement at one grade level or which^app^eared to have 
a significant relationship to^an attitude measure across both grade 
.levels^-^There are enough differences in classroom instruction between 
grade levels and between reading and mathematics instruction that we 
liesitate to generalise- across grades and 1 instructional, areas. Teacher 
performance data which can invariably be assumed valid for predicting 
.pupil 'achievement or attitude outcomes' may not exist, * 
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Summary of Observational Variables with Significant Partial 
Correlations with Two or More Outcome Measures - Pupil Events 



Outcome Measures 



Pupil Events 


2 . 


5 


2 


5 


+ Academic Performance 




(+-) 






+ Attentiveness 




( + ++) 


• 




+~ Direction Following — — 










+ Engagement 






( + + ) 




+ Participation 




( ) 






+ Volunteering 


■a 






(++) 


+ Work Habits 


J 




> 




- Conduct 










- Inattention 






( ) 




- Talking 


( ) 




— 


(----) 


0 Conduct 


( ) 








0 Engagement 


. ; (--) 






(++) 


0 Inattention 


(++++> 


. (--) 


- 


( ) 


0 Work Habits 




i 

i 




( \ 


/ 

+ Oral Reading 




(+++) 






+ Phonic Skills 


(++) 








+ Reading Comprehension 








i 


+ Number Concepts 


V 








- Number Concepts 


• 






. (+--5 


0 Number Concepts - 
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Table 51 

Summary of Observational Variables with Significant Partial 
Correlations with Two or More Outcome Measures - Teacher Activities 



Outcome Measures 

Mathematics 



Activities 


2 




5 


2 


5 


Academic Organizing 


( ) 










* 

Answering 






(+-) 


(--)' i 




Asking 
At Desk 


(+-) 








At f Board 












Checking 

Circulating 

Discussion 


(+++) 




L 






ExDlaininc 












Helping 












Instruction Giving 


(--) 




(— ) 






Listening 












• 

Question & Answer 












Supervising 


( — ) 








» 


Working with Groups 
or Individuals 
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Table 52 ' ' 

Summary of Observational Variables with Significant Partial 
Correlations with Two or More Outcome Measures - Teacher Responses 



Teacher 



Outcome Measures 



> 



Responses 


2 


5 , 


2 


5 


Instructing 


C++) 








Disciplining 






( — ) 




Teacher Help 








( 4. 4.4.x 


Ignoring 










Moves On 




( — ) • 






Negative Feedback 










None 








(-) 


Positive Feedback 


(++++> 






<+++) 


Praise 


(++> 






(++> 


Questioning 










Recognizing 






(--) 


c — > 


Redirecting 


(— ) 


i 


(+-> 


(+-> 
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Table 52A 

Summary of Observational Variables with Significant Partial 
Correlations (^cross Second and Eifth Grades and 
Reading and Mathematics - Pupil Events - * 



Outcome Measures 



Pupil Events 


Total Attitude 


l-Ol JUL V 


+ Academic Performance 


• 




+ Attentiveness 


• t 


; 


+ Direction Following 


> 




+ Engagement 






+ Participation 


- 


J ( + — ) 


+ Volunteering 






+ Work Habits 




(--) 


- Conduct 






- Inattention 




* 


- Talking 


(— ) 




0 Conduct 


> 




0 Engagement 




> 


0 Inattention 


\ 




0 Work Habits 


** 


' <— ) 


+ Oral Reading 






+ Phonic Skills 






+ Reading Comprehension 






+ Number Concepts 






- Number Concepts 




* 


• 

0 Number Concepts 


* \ , 
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Table 52B 

Summary of Observational Variables with Significant Partial 
Correlations Across Second and Fifth Grades and 
Reading and Mathematics - Teacher Activities 



Outcome Measures 



Teacher 
Activities 



Total 'Attitude 



Total Survey % 



Academic Organizing 
Answer i 
Asking 
At Desk 
At Board 
Checking J 
Circulating 
Discussion 
Explaining 
Helping 

0 

Instruction Giving 

Listening 

Question & Answer 

Supervising 

Working with Groups 
or Individuals" 



(-+) 



(++~) 



(++) 



(") 
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Table 52C 

Summary of Observational Variables with Significant Partial 
Correlations Acrass Second and Fifth Grades and 
Reading and Mathematics - Teacher Responses 



Outcome Measures 



Teacher 
Responses 


Total Attitude 


Total Survey 


Ins tructing 






Dis c ip lining t 


• 


' (+-) 


Teacher Hell) 






Ignoring. 


• 




Moves On 




(+-) 


Negative Feedback 




' (+-) 


None 






Positive Feedback 


(++) 




Praise 






Questioning 


* 




Recognizing 






Redirecting 


(++) 


t 
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Partial Correlations of Time Allocation Variables with Reading Achievement 
Outcomes 

In an earlier section we described the six time allocation variables 
derived from the. Observer Summary Report. For both reading and mathe- 
matics, the time variables are: (1) the mean number of minutes per 
student per day in each. context, (2) the mean number of productive 
minutes per student per day in each coritext, (3) the mean percent total 
minutes per student per day, (4) the mean percent productive minutes per 

/ 

student per day, (5) the mean percent gross time per student per day, & 
and (6) the me^an percent gross productive time per student per day. These 
measures provide us with the actual time spent in each context, the 
number of actual minutes that were judged to be productive, and estimates 
of the average proportion of time students in each class were in each 
context (and the average proportion of time students, in each class were 
in each context) and the average proportion of productive time. In 
determining the proportion, the reader w,ill recall that the percents 
were averaged across students for variables 3 and 4 above, and the percents 
also were computed by obtaining the total amount of time across. all 
students and then dividing this into the total amount of time in each 
context for variables 5 and 6. (See pp. 62-63 for a complete description 
of the computation procedures.) 

We $ld not use the proportion of productive time measures (variables 
4 and 6) in the computation of the partial correlations. We made this 
decision because of the relative infrequency of the "Independent-Individual" 
context, th£ "Teacher-Individual" context, and the. adult contexts. We 
attempted to compute the correlations for those classrooms in which these 
contexts were observed, but the numbers were too few to obtain meaning- 
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ful correlations. We decided instead to use the mean number of minutes 
(variable 1), the mean number of productive minutes (variable 2), the 
mean percent total minutes (variable 3), and the/ mean percent gross time 

(variable 5) as the predictor variables, 

«.*• 

In '^is discussion we will present the findings relating time 

V. - 

variables to outcomes by instructional area and grade level. On the 

s 

b^sis of the results reported thus far, we expect that the relationship 
of these variables to outcome measures will differ with respect to grade 
and curriculum area. 

Partial correlations of time allocation variables with reading 
outcome measures in second grade . Table 53 presents the partial corre^ 
,lations of the mean number of minutes with the several reading achieve- 
ment measures for second and fifth grades. As this table and Tables 54, 
55, and 56 demonstrate, the only variable with a significant correlation 
to the CAT Reading Comprehension test art second grade is time allocated 
to the Independent-Individual context, which correlates positively. 
However, time in this context at second grade is negatively correlated 
with the Reading Application test, suggesting that the types of reading 

lessons pupils use when working individually and independently of the 

» 

teachet irtay be an important factor. The correlation between the Inde- 
pendemt-Individual context in second grade and CAT Reading -Comprehension 
is positive and signif ican ^-regardless of the time variable used. The 
more, productive the student time in context, the better the achievement, 
and the greater the proportion of time in this context, „ the betteir-the 
CAT' Reading Comprehension scores. The negative correlation of thi*^ . 
context with Reading Application persists at a significant level only 
when the time variables were the actual number of minutes or the actual 
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number' of productive minutes. The proportion of time in thii context, is 
not significantly correlated with reading" app] 

While the amount olf time pupils workedj^dependently and produc- o 
tively In ind^vidfaal -contexts is cor tela fc£dwith the, reading comprehen- 



sion measures at second grad<e, time spent by the teacher coitducting a ^ 

\ - 

lesson for the entire class i^s significantly correlated with the Decoding 
Test at that level. The correlations are qi^Jte Jiigh (.44 to .50^for 



all time allocatidrnneasures. In contrast^, <thte amount of time spent in 
x the Independent-Group Contexts, the contexts in which pupils work inde- 

. - < i 

pendently of the teacher on group assignments, is negatively correlated 

with the Decoding Test at second grade. Here we have clear evidence 

that the learning of decoding skills at second grade appears to be 

dependent on the amount of instructional time during which the teacher 

is actively directing tfr^l$arning of the class. The fat:t that the 

Teacher-Group context, qbes trot show positive and significant correlations 

* / * *"* 

with the Decoding Test seems-^to indicate that instruction in second 

V 

^gj^ad^^Treading group learning situations does not emphasize decoding 

skills, but perhaps reading comprehension or oral reading skills. Since 
there were no measures of oral reading in the set of outcome measures, 

it is not possible to determine whether "there is a relationship between 

"\ t 

any of these time measures in contexts and oral readings skill. 

The fact that the correlatioris are alwatys' negative when time, spent 
in the Independent-Group contexts is correlated .with the second grade. 



reading measures presents an interesting puzzle. We might ass&me that 

* , to. 

the more time a pupil spends learning, tl^e* better the learning outcome.. 
One explanation of this finding is that, the time measures may. have ^a^ > 

> • 164 - . . 



curvilinear relationship to ac&ievement . Jhere is a range ~of timfe in 




context wj^n^J^aj^iTg^Ts optimum. -Either less or more time than that 
.at which learning is at a peak level produces poorer: outcomes. To test 
that hypothesis it would be necessary: to explA^ways of charting class- 
room achievement with respect to*time allocation in order to determine 
what^time ranges in each context for each outcome have the greatest 

learning payoff.'* 

Partial correlation of time all e gation variabiles with reading outcome 



measures in fifth gra^e ,,^ At the fifth -grade level whe^re reading instrwc-* 



tion is frequently carrieM on in Independent-Class contexts, it is time 
spent in the Teacher-Indivkdual context, which is significantly and *• s 
positively correlated with \the Decoding Test. This— finding and the""^ 

\ f * 

evidence from the second gride data suggest that if ^decoding skills are 

* \ " J * 

to be learned, they ar^.- learned in instructional contexts where teachers 

are actively, directing the learning; the more time spent, the better the , 

learning, \^itcome . At fifth, grade few pupils'need assistance with decoding 

skills. However, those teachers w^o spend time individually with pupils 



who do need extra assistance have significantly greater improvement in 
decoding than those who do^not. 

While gain in reading comprehension skills for second graders is 

.associated with the amount of time pupils spend working individually 

{ * A 

and . independently of the teacher, at the fifth grade the actual number 
of minutes and the tiumber of ; productive minutes (though not the propor- 
tion of time) spent by pupils in Teacher-Class contexts are positively 

* • ' X 

and significantly correlated with the CAT Reading Comprehension Teat. 

. : • ^\ 



We have to assume that teacher-directed reading instruction at fifth 

improved decoding and reading "^-^ 

16 5 " " ■"■ V ' " - v 



grade is' positively* associated with improved decoding and r^adihg 
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comprehension skills and that decoding skills, are most effectively , 
taught in individual rather than group or class situations at that 
grade level, * "~ 1 " » 

: r- Partial correlations of time allocation variables with mathematics 
outcomes in second grade . For second graders the most effective context^ 
time allocations for mathematics learning were the individual contexts 
Independent-Individual, Adult-Individual and Teacher-Individual. The 
reader will recall that we have reported that these individual contexts 

tg\p upils ar e wo,rfcin^o*n individualized mathematics 'assignments are 
not common in the BTES classrooms and .that most of the instruction is *, 
either by class or by group: The consistently positive correlations "of 
the individual contexts with mathematics learniil£ in second grade 
indicate that learning outcomes in these contexts are directly related 
to the amount of time students spend in them (Tables 57 through 60), 

Partial correlations of tlj^al location variables with mathematics 
Outcomes in. fifth -grade !. It i^interestifig that the -most effective -use 
of teacher time .in fifth grade mathematics was the Teacher-Class context. 
The amount of time and the proportion of time spent in this Qontexf is 
consistently and positively related to mathematics learning^ Learning 
fifth grade mathematics, li^e decoding skills in reading, appears to be, 
dependent upon the amount of .time^ teachers direct pupils, in instruction. 
Both mathematics and decoding skill? require rule and principle learning, 

and practice in applying' these rules and -principles. The more time the 

* . ♦ 
teacher spends directing these learning experiences, the better the 

learning outcomes will be. Conversely;, the time pupils spend on assign^ 

ments\ where the same activity is assigned to all pupils is negatively^ 

related to mathematics achievement (IndividualrClask context), .V 

16-6. " 1 •. "' : 
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Partial Correlations of Time Allocation Measures with Pupii Attitude 
Outc omes 

— 4 

I ~ -* * / 

I 

Second grade pupil attitude toward reading and mathematics does not 
seem to be affected by the amount of time spent in any of the instruc- 
tional cpntexts. The possible\xception to this generalization is that 
the .more time the second grade teacher spends in Teacher-Class instruc- 
tion, the more negative pupils 1 attitudes about learning become (Tables 
61 through 64) . 

In fifth grade, time devoted to Teacher-Class instruction in both 
reading and mathematics is positively and significantly correlated with 
positive change in attitude. It is interesting to note again the impor- 
tance of time spent by pupils in 'teacher-directed instruction in decoding 

One possible inference is that 
grade 'pupils who learn skills in word attack or mathematics have 
more positive attitudes at the end of the year. 



skills and inMathemat iss achievement* 

T 

fifth grade pupils who learn skills in 



The survey measure reflects self appraisal of competence in school < 

\ - v. 

and other aspects of self differentiation. Several of the time variables 
are significantly related to positive changes on the survey measure. At 
the secdfwiL^rad^e level, time in individual contexts in reading is nega- 
tively correlated with the total survey measure while time in group 



contexts is positively correlated. These data suggest that the more a 
pupil's learning experiences differ from those of his classmates, the 
poorer the se^^B^n-cept outcome (even though time in these contexts is 
positively rela^J^Ho achievement). This generalization is supported to 
some extent by the correlations o^t-tm^in Teaphr£r-Class contexts for 
second grade mathematics. 
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erviev'pf fcfre Relationship of Context Time Allocation Variables 



\to Achrgggment and^ Attitude Outcomes 

^X^In the following^bable (Table 64A) we summarize* for reading 

\ 

and mathematics, those context time allocation variables which have 
a significant relationship to two or more outcome measures. Within 
the cells of th^ tables are listed the particular contexts for whic 



the time measures^ (listed in column 1) showed a significant 

relationship to achievement. The direction of t^e^relationsh 

* 

among the various context time allocation variabl£S~and the 
Achievement tests with which they^were correlated' is indicated 
by the use.of "+" a^d signs after the context designation. 
For the summary with the attitude measures (T^ble 6#B) ^ included 
those contexts which had a significant* correlation^aeross both 
grade levels within ea'ch of 'the two attitude tests. 



\ 
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Comparisons Between Teachers With Better Than Predicted -and Poorer 




Than Predicted Achievement Outcomes Using Crosstabulatipns of 

'\r^r...... — ; . 

QbservationH/Categories - v / ' 

JJJie ^a^a^thS^^ar illustrate' the verie.ty^of information, obtained 

through observations , which can be used to measure teacher performance 

and which' has a potentially va^id ^relationship , to achievement and a\ti- 

tude outcomes^^-Eltcept for r theXtime allocation, data, the presentation 
* j - — * * * / 

of % results has treated theVsets of 6t>servation variables as though they 
were items iri a category observation system./* The sets of variables, or 
the categories, af^t ^ instructional context, teacher activities, pupil 
events and teacher responses. The interrelationships apiong the categories, 
can be conceptualized i n j^ a nim^er of ways, but an antecedent/consequence 




approach focuses on instructional contexts £nd teaching activities which 



V 



.result iri particular ty^es fcjf pupil eveht^, which in^urn , elicit 
particular teacher resp6ns.es/ The APPLE Information System permits th 

' V h ■ 

retrievals of a all possible combinations .of teacherXEMP^l observation 
varijg^es with the relevant frequency counts! ' 
, . * One of our % underlying assumptions abQ'ut teacher perfofmancfe char- 
acteristi'cs in elementa.ry education programs is that the way the teacher 
.organizes instruction determines the'^exteftt^t^ which pupils in the . " •» 

^ 'classropta learn under the teactier's direction or* function indepetidently 
of 4 the ^teacher T s instruction. *We believe tR£t this organizational r ^ 
pattern can^affect learning outcomes. We hav^ demons trated that there 
• dre gseat differences in the ways that reading and mathematics instruction . 
v ,\ ' " afo. organized in the second _|rade, and that «both jeading and mathematics 
iR^ri^Qtton 4iJ^T'v8reatly be twefen -.second grade and fifth grade. 
, Our data show th^'d^unreasonable td specify gene*r^kteacher perfor-.. 

1-7 Sf • ' 




9 

ERIC 



V. 



A 



mance variables which would uniformly affect reading and mathematics 
outcomes or that would have comparable applicability for the analysis 



of instruction across grade levels. Furthermore, predictive validity 
studies of the relationship between the observation variables and 
attitude outcomes provide other important findings about teacher 
performance. Teaching characteristics which affect achievement ojtt- 
comes positively do not always have a* positive relationship *tp acti- 



t.ude outcomes. We- conclude on the basis of the work presented in this 

* * * 

* ' 1 * 

report that teacher performance variables and pupil outcomes miisj: be 

' ' ' / 

studied by grade level and by specific attitude and achievement out- 
comes, it} specific suyfecp areas within grade levels, furthermore we 
believe that an adequate study of classroom observation variables must 

r 

treat gr&^igs^ef -variables rather^han individual variables ^s the 
units of aifelysis for greatest understanding *of teaching-learning 
patterns. * " 

, In order, to provide a basis for developing additional hypotheses 

about observable teacher/pupi~l~-ctassroom characteristics /which are 

e ** * 

associated with greater or. less than predicted gains' ~in /reading and 
mathematics, We ideritifiedf If each grade level and in each subject , 

area the five teachers whose* outcomes were better and the five teachers 

c / \ 

whose outcomes were poorer than would have been predicted by fall tes 
•scores. The five top and' the five bottom teachers we're located by 
performing ^egression analyses for second grade reading outcomes, 

second grade mathematics outcomes, fifth 1 grade reading outcomes and 

i / « 

fifth grade mathematics outcomes. (Volume I descri/bes the procedures 

* v " A / 

used to identify the teachers for. the crosstabulations. As reported 

• ' / * 

in Volume I, hodfever, there i^ a discrepancy between the number of 
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teachers identified. While the top and bottom ten teachers wer$ used 
for the analyses of Volume I, we chose instead to do our analyses with 
only the top and bottom five. This choice should not lead to contra- 
dictory results, however, since selection of the five most extreme - 
rather than ten should make more pronounced any differences observed 
between the groups~of teachers* > Table 65 presents the distribution 
of the combined second and fifth grade top and bottom teachers wlth- 
respect to SES and location* The reader will observe ^that the class-* 
rooms of the top^ and bottom teachers were distributed over all loca- 
tions and over all SES levels with the exceptions that there were,, no 
top performing classrooms in the low/ low-middle class suburbart areas, 
^.and no top performing classrooms in middle SES rural locations. 



addle SES rt 
it the cj/assi 



Overall then, we can tte satisfied that the cj/assrooms with* better and 

' . . . • J ■ - " 

poorer than predicted outcomes were distributed across a wide variety 

• i ^ » * ' * 

of schools and were not associated with a particular SES level or 

* ' » ■* 

geographic location. We can now ^roceed^with the review, of the findings 

with jrespect to teacher/ pupil interaction analyses. In the review to 

be presented, we yill not detail all of the possible crosStabulations 

of categories in o^r^discus'sion. Instead we will illustrate, using 

the example of second grade reading, the ways in which a reader might ^ > 

use the crosstabulatipn tables to develop hypotheses about classroom 

(teacher) performance. « " - «\ * 

Crosstabulations of instructional contexts and teacher ^activities 
: " : : : — ^ 7- 

using the example of second grade reading . We will first examine data 
A which test the assumption that teacher and pupil behaviora in differ- 

ent instructional contexts provide performance data of potential 
0 validity for analyzing outcomes in second grade reading.. Table 66 " ~ , 

presents the crosstabulation of teacher activities and instructional , . 

er|c 181 . ; i 



V 

Table 65 \ 

Distribution of Classrooms with Better than and Poorer 
than Predicted Outcomes in Reading and Mathematics 



Location 


SES 


Low - Low-s-Mlddle 
Top Bottom 


Middle , 
Tod.— ""-aaStom 


Middle-High - High 
t Top Bottom 


Rural 
Suburban 

> 

Urban 
Total 


3 ". ■ 1 - • . 
/ - ' 

o < ^-2^ 

2 - 2 , 
5 6 


0 '4 

\ ~ : 
2 3 

4 , 3 
6 " 10 


' 1 '1 ,11 
4 2 13 
4 1 16 
9 ^ _ 4^^ 40 



contexts. . A brief explanation about how this table is organized will 

0 

aid the reader in understanding how this .specif ic table was interpreted 
ani will also provide a guide for interpreting subsequent tables. 
Reference to the column percents provides information as to how top 
and bottom teachers differ with respect to the frequency of contexts 
associated with a particular teacher activity. As an example, take 
the activity /Circulating 1 / from Table 66* Note that* for the top , 



teachers' .two contexts are?observed when the "Circulating" activity is 
seen ("I-Class" in* 85.7% of " th6 instances ajid • "J-Individual^_in th^ ^ 
remaining 14 \ 3%) . For the bottom- teachers the activity never occurs, * 



regardless of coatext. Similarly, [reference to the row percents. shows 

V 

how the top and bottom, teachers differ with respect to the frequency 
of m activities observed with a particular context* For example, take 
the "A^Group" context* The percents associated with the teacher 
activities observed for the top teachers in the M A-Group M , context are: 
11.1% for "Academic Organizing", 2.2% for "Instruction Giving' 1 and 
86.7% for "Working With" . For the bottom teachers, three of the 15 
activities (two of which wfere the same as for the top teachers) occur 
within the "A-Group" context. Tha "Percent Column Totals" indicate . 
percentage occurrence of each activity compared with' all athers, 
regardless of context, and the ''Percent Row Totals," show percentage 

o 

occurrence of each context compared with all others, irrespective of 
teacher activity. 

"Having given examples to illustrate the basic organization and 
use of these crosstabulation tables, we now proceed to discuss. the 
important hypotheses generated. In the first table presented (Table 
66), the "Teacher-Group" context arid the "Adult-Group" context are 
more frequently observed in the classrooms of the teachers *in the top 

183 ' ' : . 
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^ y 

group, vhil§x£ne "Independent-Group" context is less frequently 
observed. We Ti&ve^ already pointed out that the "Teacher-Group" and 

ndependent-Group" conX^xts are the most common instructional modes 
for second grade reading. However, the table shows that the top 

teachers\ along with other adults in the class, are responsible for 

\. ♦ 

instruction*^ over 50 percent of all of the target pupil' events . The 
"next question a^s: Do teacher activities in different contexts vary 
between the top and >pttom teachers? Table 66 illustrates the import- 
ant finding that the t<u> ( teachers use a greater variety of teaching 
activities than the bottXm teachers, and that there are resultant 
dif ferences y in the relative frequencies of these activities. The 
"Working With' 1 activity is most frequent for both groups of teachers, 
but the top teachers are observed more frequently working with pupils 
than are the bottom teachers. When top teachers n Work With 11 pupils, 
the "Independent-Group 11 and "Teacher-Group" contexts are equally ^ 
dominant as instructional patterns. while the bottom teachers "Work 
With" pupils predominantly when the pupils are organized independently 
in groups. Teachers of pupi^with poorer than predicted outcomes are 
observed helping pupils more, spending more time organizing ^the 
lesson, and answering questions more often. Teachers of pupils with 
better than predicted scores are observed more frequently asking 
questions, checking pupils 1 work, giving instructions, listening, and 
conducting question and answer sessions. , *S\ 

Crosstabulations of instructional contexts and pupil events using 

y 

the example of second grade reading . Pupil behavior iS/also observed 

to vary with respect to contexts. Pupi]^ in the top teachers 1 classes 

have more observable reading' events (see Tabie 67), such as' "+ Oral^ 

Reading", '"+ Phonics Skills" and "+ Reading Comprehension". f, + Oral 

±85 



158 



• I * 



• - 4*1 



O C 
. O -« 



6 • a 
-•to 

^ v> 
» > e 

ma 

* • -« 

; s:; 

> V I 
M « O 

— m •» 

-I I- 3 

<*•. O 

2 5/S 

». X .* 



••|3C»33TOI - 



V 




3- 



-ERJC 




5 



© C 



o *•} 
<n *» 



2 " 



o *1 - 



I 2 



5 « 



186 



Reading" occurs more in the "Teacher-Group" context tHin in any other 



context. ' "+ Phonics Skills 11 are observed for both grot ps of teachers 



but-^^Phonic s 
dult-Group" 



in the "Independent-Group" and "Teacher-Group" contexts 
Skills" are. observed only for the top teachers in the " 
and the ."Independent-Individual" contexts^ It is of interest that * 
there are no reading performance events observed with adults leading 



the instruction in the bottom , teachers' classes, gupils 



in the Uottom 



t^Sachers 1 classes are more often seen manifesting general academic 
skills, following directions, and demonstrating 'good work habits than 
are pupils in classes. oINthe top teachers. The two groups of cl£ss^ 
li)fer with respect to the extent of teacher manage- 

and evidence 



rooms appear to di 



ment of instruction, the variety of classroom activities, 
of reading skill learning. Of considerable importance ia the finding 
that top and bottom teachers do not differ with respect to the frequency 
of observed negative pupil behavior ^ , \ln * fact, pupils in the top 
performing' teachers 1 classes manifest slightly more negative behavior 
_when the frequencies of "-^(J^nduct, " "V Inattention, 1 ' and "- Talking" 
events are combined. 

Crosstabulations of instructional contexts and teacher responses 
using tl\e example of second grade reading . Table 68 shows that the 
bottom teachers more frequently fail to respond to pupil behaviors 
than do .the top teachers, though "None" is the most frequently ob- 

served teacher response to pupil behavior ov^r all classrooms. The 

/ 

bottom teachers use "Discipline" more often than the top teachers, use 
more "Praise", and employ^a smaller variety of teacher responses. Top 
teachers employ more interactive types of management such as "Instructing, 
"Questioning," "Recognizing," and "Redirecting" across the several' 
contexts.. - " 
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Crosstabulations of pupil events and teacher activities using, the 
example of second grade reading . The path analysis proposed for 
conceptualizing 4 the relationship' between tea cheiP performance variables 

^^^^ * \ 

and' pupil outcomes postulates that teacher activities generate \pupil 

" • - • • \ ' ' ' ' ■ J 

behaviors and pupil behaviors generar^teadher responses. Wtfile 'feactv 
of these categories of variables may be considered aspects qi teacher 
performance, the feasibility of using this conceptS&ignodel to analyze 

* the reciprocal relationship between teacher and pupil behavio: 
supported by fche cros^tabulation of teacher activities and pupil 

events presented here and the cross tabulation of pupil events ar 

- **■ y / 

teacher Responses to be discussed below. Table 69 proyid^s^compara- 
tive "information about the-'teaching activities in which reading skills ^ 
are most evident. In the classes of .the top teachers, pupil' events in 
"+ Oral Reading" occur in academic organization activiti/es, in -response 
to teachers -asking questions, during discussions and while teachers 

* are listening, as well as in sessions in which teachers, are working* 
with pupils. ^On the other hand, all of the "+ Oral Reading" events in ^ 
the bottom teachers' classes occur during ."Work With" activities. A 

similar distribution of "+ Skills" events occur across instructional 

** '*» ' * * * * 

n activities for the top performing teachera, while all of the phonics 
»' events in the bottom teachers 1 classes were again in the "Jfoifk- With" 

* activity. Second grade teachers tyhose pupils showed bfetter than • 

' , \ 
predicted achievement in reading utilize pupils 1 reading skills ; in j ^ 

many .more type^ of instructional Activities than d6 teacliers whose 

pupils have lower 'than' expected achievement outcomes. * 

' The overall frequency of -* f + Engagement" .events is similar, for 

■* • i. 

both the top and bottota teachers. However, in activities other than 
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"Work With", being engaged in the learning task was observed to be 
esent when the top teachers were, "Asking" questions, "Checking" 
1 werk, "Circulating" around the room and "Supervising". The "+ 
ngagement" events for the bottom teachers were observed instead in 
"Academic Organizing", "Answering" questions, and "Helpiiig" activities. 
Pupils are engaged in the top teachers 1 classes when the teachers are 
actively involved in instruction.* In the bottom classes pupil engage- 



ment is observed in teaching activities associated with introducing, 

* - • t < 

conducting the lesson, and helping pupils who are^having difficulties. 

Additional evidence of more active pupil participation and greater 
diversity of teacher activities among the top teachers c'an be noted in 
the "+ Volunteering" events. There are no "+ Volunteering" events 
observed in the bottom teachers' classes, while a small portion" of the . 
total observed pupil events in the top classes are "+ Volunteering". 1 
These events are observed 'most frequently in the "Work With" activity, 

- F ? 9 

but they also^ occur in "Academic Organizing", "Asking", and in "Question 
^nd Answer" sessions. These data* contrasting tesfcher activity/pupil 
,event combinations suggest the important hypothesis that crucial 
second grade teacher performances are engaging the active participation 
of the pupils in reading lessons and integrating a variety ^of teacher 
activities into reading instruction. . - 



The ^validity studies of teacher activities, in second grade reading 
reported significant correlations with reading ^achievement oyer all 



^BTfi^-clas 



lassrooms as a group (see Table 40). The data, showed that 



"Academic Organizing" and fT Asking". are negatively correlated with some 
measures of reading aphidyementr outcomes. By using the crosstabulation 
of teacher activities * and. pv^il events, we can point out some of the 
possible explanations fr>r .th|se negat^ye relationships., The percent 

191 



of "Academic Organizing" activities is slightly higher for the poor 
^ teachers than for the good teachers. However, there were* more "Asking" 
activities for the good teachers. By examining each of the pupil 
events observed within the "Academic Organizing" activity we see that 
\the pupil^eyents in the bottom teachers' classes a£e "+ Academic 
Perfprmance", "+ Engagement", "+ Participation", and "0 Conduct". The 
effectiveness^ of the' top per£gjrming teachers in using "Academic 

^Organizing 11 £ftae is amply shown by the incidence of pupil events in ^ 

*\ \ ^A 

l:lie categories "-i-^Aeademic Performance", "+ Attentiveness", "+ Pirection 
Following", "+ Engagement", "+ .Volunteering", **+ Work Habits", "+ Oral 
Reading", and "+ Phonic Skills" as well as the 'frequent occurrences ojc 

Inattention", "0 Corrduct", "0 Innattention" , and "0 Work, Habits". 
The considerably larger proportion of "Asking" activities for the top 

t 

teachers is contrary to the findings for the second grade teachers as 
a group, for which the correlation of this activity with outcome was * 
negative.* Nevertheless , ^we catV postulate that when "Asking* 1 is ac- 
companied by active pupil responses, such as lf + Direction Following" 

0 

and "+ Volunteering", the effect on outcome may be positive. 

Cross tabulations of pupil events and teacher responses using 
tjie example of second grade reading . Nearly everyone would expect 
that teachers with the best reading achievement outcomes would liave 
few, if any behavior problems in their classes, especially during" 
" reading. The crosstabulations of pupi^l events and teacher responses 
(Table 70) show that th^re are* at least equivalent frequencies of 
classroom behavior problems as reflected in Conduct arid Inat;- 
tention". The crosstabulations may suggest Why the v incidence- of 
negative pupil behavior is not related to reading achievement outcomes 
in the' top .teachers 1 classes. Earlier we pointed out that there were 

-s. 1 

■ /M92 .•< ' (■ 



variety* of teacher responses in the top classes.' Table 70 
provides clear "evidence^ that there are not only a greater variety of 
teacher* responses, but that top^te^chers 1 responses' to particular 
types pf 'negative pupil behavior vary as w^iL. . . The -top fe a rhera 
respond to Conduct" pupil behavior with "ignoring", "Negative 
Feedback", "None" and "Redirecting 1 ^. In the bqttom performing classes, 
teachers respond to this behavior with ^Disciplining" and "None"/ 
Inappropriate pupil behavior* ±s common to all classrooms. Top scoring 
teachers presumably do not disrupt their teaching to discipline an 
inappropriately behaving pupil* They maintain .control of the situation 
by ignoring the pupil and moving on with* the lesson'. ' , 

Teacher responses such as "Positive Feedback", "Instructing", and 
"Redirecting" were also related to achievement and/attitude outcomes. 

However, the relationships were both positive and negative. Conse- 

/ 

quently, one must teturn to the data such as that .reported on these 
t . ■ • 

* * * * • 

tabbies to -understand the interrelationship between pupil and teacher 

variables as combined predictors of outcomes. 

This explication of the *crosstabiilation data for the second g*cade 

► > • '* 

t easier s with the top and bottom outcomes in reading achievement has 

been offered tp show t^e reader one way that the wealth of data .col- 

lected in the APPLE naturalistic observation system might be used. 

» 

Tables reporting similar results for second grade teachers vith -the, 
•* * * * \ 

best and poorest mathematics outcomes follow (Tables /71-75) , along 
with tables reporting the data for the toj\ and bottom scbring fifth " . 

grad^ teachers in reading (Tables 76-8Q) and iiv mathematics (Taibles 

. 1 * , • I * • * \ 

81-85). From a careful review of all of the cells, in wMch inter* £ 

active data are available, 'a future' task will* be to identify the 

frequently occuring teacher activity/pupil event combinations and 



19 



pupil event/teacher response copbijnations and t to use them as independent- 
^variables in the prediction of pupil outcome^. The ^'APPLE Observation 
System categorizes Naturalistic observations and provides data which, 
when- analyzed as category data", can produce an. initial screening of ' 

i .1. 1 ... 

the most salient classroom variables from which to in£er teacher 




nee characteristics. The tables reporting Jthe crosstabula- . 

tions are excellent evidence for the fact that; many of the critical 

teache'r performance characteristics are^ho't; identified by talli&g of 

the overall. J&gegoetrcjr of^hElr^GGurxence^^Onl^ , when teacher activity 

variables- are analyzfecM*ith respectr to the 'consequent *pupil behavior 

and the subsequent teacher behavior is- it possible to understand. 

differences between teachers with the best an<} poorest outcomes in 

f ... 

'particular achievement areas. * - *, 
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Summary of w the findings from the APPLE Observation^ 

* i s 
The behavior recording System employed in APPLE observations provided 

* 4 \ 

data on pupil behavior' in reading and mathematics § instruction, teacher 
responses to pupil behavior, the instruct iorial contexts for both instruc- 
tional areas at eabh grade level, and. the teaching<acti^ities wfiicn 

. ' ' " • > . \ 

typified teacher behavior at. second and fifth grades in reading and v 

\ 

mathematics lessons^- In addition, time duration dat-a^ were available^ 
These data ref letted - time allocation ;of instruction in the several \ 

contexts which were common to the BTES classes. In this section, we \ 

< 

will review the several generalizations we have made. T)i4se generali- 

8 % \ 

zations, summarize the results of our ^investigation, and provide, informa- 
tion about second and^fifth grade teacher performance characteristics as 
well as the validity of these characteristics for predicting children's r 
learning 

\ 

Description of ''Pupil and Teacher Behavior in. BTES Classrooms 

From the discussion of the statistics describing cfentral tendencies 

and variabilities of these data across classrooms, instructional areas, 

, anci grade levels, several statements appropriately summarize- the results. 

1. Differences in incidence of pupil events are a function of^ the 

instructional area, the instructional context and the teaching activities. 

• * - t . 

This finding means that one 'must expect to find different types of pupil, 

behavior and differing frequencies of pupil behavior, depending on 

•whether the child is a second or a fifth grader, whether he is involved 

in reading or mathematics lessons, whether he is working^with the teacher' 

or independently,' and- what the activity of the teacher is. 

■ 211 * 
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^ '2. In general, we can conclude that fifth graders will be expected 
•to spend more of their classroom time working independently of the 
teacher for both reading and mathematics activities than will second 
graders. While there is considerable variability , among second grade 
teachers in the use of different instructional contexts, secortd grade 
pupils wili$be involved in more teacher-directed instruction than fifth 



graders. "1^ * j 

3. »The quality of teacher responses to pupil behavior is very 

similar across both grade and instructional area. The response of 

V* ■ 
"None" to observed pupil behavior occurred afcout two-third^ of the time. 



we do not mean to imply that a teache^ response should accompany every 

r (- 

qbserved pupil behavior; rather the data indicate the probability pf fhe 
occurrence of any teacher response t<5 pupil behavior in the classroom. 

4.- In 'organized instructional periods such as reading and mathe- 
matics, conduct -problems and need for- teacher disciplining are not 
common. This finding suggests that supervisors, and>others who visit 

s 

classrooms, cannot rely on the absence of conduct problems as good J. 
evidence for teacher competency* ^Inappropriate behavior during ac&demic^ 
lessons was relatively rare. 

Descriptions of Time Allocations' in Contexts in BTES Classrooms 

When we examined teacher performance with respect to the ways 
teachers distributed classroom instructional time in different contexts, 4 
these generalizations f.ollowed from the descriptive statistics. 

1. Second graders spend v more total minutes in reading instruction 
than do fifth graders, and this is plimarily reflected 'in-qnore time 
spent in group instruction and time spent being taught by other adults 
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Independent-Individual contexts, those contexts in which pupils were 
working on an individualized assignment different from other pupils and 
independently of the teacher, were more common in second grade reading 
than in second grade mathematics and also more common than in fifth 
grade instruction. 

2. One surprising finding was that the average proportion of pupil 
productive time in individual contexts in reading was relatively small. 
Few pupils receive such instruction and of those who do*, the average 
percent of productivity as judged by the observers'- was lower than the 
percent of gross productivity. This finding implies that the teacher's 
Choice of this context for certain pupils is^not always a wise one. 
Only those pupils who can manage such a context for * significant periods 
of time (probably greater than 5 to 10 minutes) are likely to profit 
•from it at all." • 

v 3. The primary difference in time allocation in mathematics between 
grades was the relatively greater proportion of time spent in Teacher- 
Class contexts at' second grade. 

4. The time spent in Independent-Individual*, Adult-Individual, 
Teacher-Individual and Aduit-Group contexts is nof generally productive 

time in mathematics when the percent of productive time was averaged 

. : . ' ' ■ . ' 

acrof^^puj^ls . However, when the gros^ productive time was computed by 

ft tv * 

finding the proportion of productive time by classroom, rather than 
averaging across* pupils, the' productivity rate for these contexts was as 
high* as for other contexts. This finding, along with the similar result 
reported for second grade, indicates, that only a few pupils work produc- t 
tively in individual contexts. Therefore the average pupil productivity 
is lower -in individual contexts than it is in settings where instruction 
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is by group or .by class. However, individual contexts are appropriate 
and productive for sbme pupils. Thus, when one looks at- the rate 
obtained by dividing total productive time for all target pupils by 
total time in that context for all target . pupils , the productivity is 
higher. 



Validity, of Observational Variables for Predicting Reading Outcomes 

I , 

The great variability in pupil and teacher behavior between grade 

j 

levels and between instructional areas leads to the preliminary conclu- 

sion that ve can, not expect^to find a set of teacher performance variables 

f • t 

which will be valid for predicting learning outcomes across grade levels 

and instructional areas. The resul'ts of the partial cotrelations between 

< _ 

observation variables and ° the reading achievement measures support this 

i 

^^tia^ impression* ^^cltne^f r *th«%^i'eralizations that we can \na\e on th^ 
b^sis of correlations between observation variables and reading achieve- , 
ment are: . ^ 

1. Talking out inappropriately, in classrooms during reading instruc- 

* 1 

tion was uniformly related to poorer reading outcomes. There was also 
an inverse relationship between the observed frequency of pupil behavior 

V/ 

which was not consistent with teacher expectations (but not necessarily 
serious enough % to warrant teacher intervention) anti achievement outcomes. 

2. Evidence of positive achievement in phonic^ skills was positively, 
correlated with outcomes for second grade and negatively correlated with 
reading outcomes for fifth grade. This suggests that teacher instruction 
in jsjKJfiics skills enabled second gyadjsrs to improve their achievement". 
However, it appears that teacher instruction in phonics at fifth grade " 
was directed at pupils with 4 earnin 8 difficulties and perhaps was not 

the most effective means by which to improve learning for all pupils in 

/ ' 
» * . 

the classroom. 2jL<2 



ERIC 



187 

.\ ■ 

3. Conversely, evidence of positive achievement in oral reading " 
was positively correlated with, outcomes at fifth grade, but not at 
second grade where oral reading is regularly observed for al^ pupils in 
reading instruction. 

4. Witli regard to teacher activities, we offer the tentative 
hypothesis that the more the teacher is involved in organizing the class 

.for instruction (as evidenced by negative correlations between activi- 
ties like "Academic Organizing, " n Asking Questions, 11 "Discussion, " and 
"Instruction-Giving, 11 and reading achievement) and the less the teacher 
is involved in "Working With" pupils and "Checking" to see if they are 
understanding their work, the poorer the end of the year reading achieve- 
ment. We do not intend to imply that teacher activities like "Academic . 
Organizing,"' "Asking" quest ions^and so forth are not important teacher 
performances. However, as the frequency of these activities increases, 
the frequency of actual instrcic^ion decrease^,* Consequently there will 

be reduced opportunity for reading instruction, resulting, in less , 
*- * 

r 

improvement in -reading at the end of the year. ; 

Validity of Observational Variables for Predicting Ma theiQa^trics^>ut comes 

In comparison to reading outcomes, a di^J^^^set oi observation 
variables were. valid for predicting mathematics learning. The following 
conclusions follow from the analysis of the validity of tjie observation 
variables: • ^ 

1. second grade mattffemjatics lessons during which there is a 
high frequency of inappropriate conduct, obvious inattention to class^ 
room work , and talking out of turn, there is* more likelihood^tha^t \ 
achievement scores in .mathematics will not increase as predictedfat the 
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2. Though noticeably 'Inappropriate behavior did not characterize 
fifth grade pupils in mathematics assignments, the frequency of inatten- 

r ^ 

.tion to task^which was 'not disruptive or particularly noticeable was 
uniformly negatively correlated to mathematics learning. 

3. The frequency of Independent-Group and Independent-Class 
contexts was negatively correlated with achievement outcomes at both 
grade levels, suggesting that classrooms in which pupils were working 
independently of teachers most of the time were those with* poorer 
mathematics achievement. 

/ A. A surprising finding was that teacher instructional activities 
were observed with less frequency in mathematics than in reading. Only' 
"Asking" had a significant correlation with outcome and it^was negatively 
correlated. The tentative conclusion is that teachers who spend time 
asking questions are those whose lessons ,are not understood or whose 
assignments are at an inappropriate level of difficulty. " 

5. Six out of -12 teacher responses at fifth grade and thn^J^f 12 
at second grade had a significant relationship to mathematics achieve- 



ment. At second grade, teachers who employ disciplining, recognizing, 
and redirection have poorer than expected achievement. At fifth grade, 
.giving help, providing pqsitive, feedback, and praise" are positively 
correlated with achievement. Recognizing and redirecting are negatively 
^orrela^^Lth mathematics learning at fifth grade. " ^ 

Va'lidlty-of Observation Variables for Predicting^ Attitude Outcomes 

f A most interesting finding was that teacher performance variables 
which have a valid relationship to pupils ^learning may have an 
inverse relationship to changes in pupil's atjtit-utle^toward learning 
and toward themselves. " 
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1. Different sets of appropriate and inappropriate pupil behavior 
were correlated with reading and mathematics attitudes and for second 
and fifcth grade. We have to conclude, therefore, that it is unrealistic 
to assume th^t- one can define an omnibus set of teacher performance 
variables, which, if present in a group of teachers, will guarantee 
positive learning outcomes as well as positive attitudes. 

2. With respect to attitudes toward reading and mathematics, 
second grade classrooms in which target pupils were observed to be 
positively engaged' in the activities of the class and. in which tttey were 
not misbehaving or talking Out inappropriately were those with the most 
positive attitudes at the end of the. year. 

3. Since the Pupil Suuvey items ask questions about how children 
judge their abilities with. respect t& peers, it was ndt surprising that 
some pupil event variables evidencing appropriate pupil behavior had k 
negative correlations with the survey measures. In some clssrooms, 
positive participation of target. pupil 6 s may cause pupils to contrast the 
adequacy of theii; performance unfavorably with that of their classmates, 
resulting in a negative correlation between some positive behaviors and 
*the survey measure. 

4. m At second grade, "Academic 'Performance," "Attentiveness" apd 
"Direction Following" had a positive relation-ship to the survey measure 
suggesting an age .difference in "the implications of these types oi pupil 
behavior on attitudes toward^sel^f worth. 

5. There seemd to be no significant relationship between the 
incidence o,f the instructional contexts' and attitudes Across either 
grade level or instructional area. Attitudes, therefore, are more 

dependent on teacher and pupil behavior in the classroom. 

/ 
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6. The teacher activities Wh><rfi are significantly- Delated to . « 
attitude and are always positively correlated with attitude and survey 
outcomes across ins>rtjc tional area are: 1 "Checking," '"Listening, M 
"Supervisd^^T: those with unif ormly^signlf icant negative Correlations 
are^"€irculating" and "Instruction ,Gwing!i^ --- • „ I 

7. Teacher responses to pu^lr^beh^vior had different patterns of 
relationships to attitudes according to whether teacher responses were 

* ob^^ye^-dtrTeading or mathematics and whether pupils weire second or 

" V v. , ' 

- 

fifth graders. The data indicated that the incidence ot disciplining 
an 4^egajt^ve feedback in a classroom cannot be judged jto be always nega 
tively correlated*. to general attitades ifx the classroom. On the^other 
hand, the extent of positive feedback was alwa^s_p.ositively related^fb 
altitudes when the partial correlations ;were significant. 

8. from the analysis of teacher activities, aii<i responses, we 
tentatively infer that teacher behavior which >cai;J.s attention to xhil- 
dren, and perhaps -places them in an unfavorable, comparison with their ' 
classmates, may' have a negative, affect on attitude*,. even 'though these 
same" teacher behaviors > have, a* positive relationship to achievement 

outcomes. ' , ' * 

r * 5 1 

/ , ' . • . 

Overview of Relationship of Observation .Variables to Achievement and 

Attitude Outcomes ' * v , v 

. the criterion of retention of 4 an observation, variable as .a poten- 
tial measure of teacher performanc^was a significant relationship with 
two or mare outcome measures^ The criterion of retention for altitude <\ 
outcomes was- a significant correlation ^with an butcome at bbth 
grade levels. Using thdse criteria of retention, onfy* a handful 'of 
APPLE variablel could not bi considered pbt^nfialTy .valid teacher 



performance measures,^ However, there are enough differences between 
grade levels and between reading and mathematics instruction that we 
cannot generalize across grades and instructional areas regarding which 
teacher 'performance data will always be valid for predicting pupil 
achievement or attitude outcomes. % * 

Time Allocation Data as JPxadietors of Learning and -Attitude Outcomes 

^ — . -A, 

The lengthT^f- time pupils spent in some" instructional contexts 

relates to achievement and attitude outcomes. Some of the interesting 
findings were: 

1. The more tij?ie 4 pupils spent in individual contexts in second 

'" * 
grade reading, the better their performance in reading comprehension." 

. r *~ 1 

2. While the amount of time second grade. pupilsj forked indepe 



Den- 



dent ly -and productively in individual contexts was correlated with the 
reading comprehension measures, the amount of time the teacher speat 



conducting reading lessons for the class was significantly correlated 
with the Decoding Test. The data offered clear evidence that the 
learning of decoding skill§ at secondv grade was dependent upon the 
amount of learning tdme during which the teacher is in charge of class 
instruction. . 

3. The amount of learning time in. context does not always hav^ 
pp.sjMvp relationship to outcomes.* An example was the negative correla- 
tions between . 'the Independent-Group context and the reading measures. 

( v t 

Time measures may have a curvilinear relationship to achievement. Some 
effort to determine the -amount of time which is Jfche optimum for learning 
in- each conteict would be an excellent way to extent our understanding of 
the duration of classroom experiences and learning. 
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4. Fifth graders who spent more time in contexts where teachers 
are in charge of instruction have higher score § v nn rrj 




measures . 

5. The most effective context time allocations for mathematics^ 
learning were the several individual contexts. Even though individual 
contexts were common to only a few BTES classrooms, the data indicated— 
that the more time second guaders^sp^flC lTt--iu4^idual contexts, and the 
more this time was productively spent, the better their^^thematics 
learning.^. ^ 

6. While time spent in individual contexts in mathematics was 

important to second grade mathematics achievement, the amount of t^ime 

/* * 
pupils were taught mathematics by the teacher -in fifth grade classes was 

Significantly related, to outcomes. 

7. ^Both mathematics and decoding skills require rule and principle 
learning and practice in applying these rules and principles. The mor^ 
time the teacher spends directing these learning experiences, the better 
the learning outcomes. 
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Interrelationship of Pupil and Teacher Observation Variables 

In order to illustrate ^the importance of studying the observational 

data as sets of interdependent 1 pupil .and ■ teacher characteristics, we 

prepared ..tables reporting the frequencies of occurrence of pupil behavio 

in contexts,- teacher activities in contexts and teacher responses in 

contexts. These tables provided an excellent overview of the kinds of 

pupil' and .teacher behavior one is likely to observe In the^^contexts .' 

Since* the tables cont rasted observational data for the most: and the 

— ■ f . 

least successful- teachers in terms of predicted learning outcomes, they 



provided an" interesting set of additional descriptions of -important 

* J. 
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Teacher performance measures. One additional set of tables reported the 

irent types of teaching activities, T'aqd another 



ivior in: 



\ 



jreported the teacher respons 



The APPLE In J 



ich were associated with pupil events, 
stem ma{ce^ pfcssil>le the retrieval and 



analysisof^any =: ^mbijai^ion of pupil ancplr^a^^ier beltav^or variables. 
With the availability of such coraputer^programs and th^iojfolna^t ion 
offered in the tabj.es reporting the crosstabulationsy an important^fce^ct 
step in > understanding teacher performance characteristics' wilT^be^ 
study of the interactions of pupil and teacher characteristics. The 
goal would be to stji 





*e have done 

extensive overview of pupil and teacher behavior in the BTES classro 
Conc lusion 

In general the {lata reported hfere support the use of a behavior 
recording system as a valicr^tpol for, studying teacher performance chara£- 
keristics. The APPLE system makes the fewest a_ priori assumptions about 
the nature of teacher; and pupil behavior, can describe pupil and teacher 
characteristics with considerable fidelity, and identifies those pupil 



and teacherx behaviors 



attitude outcomes for 



which are significantly relat^ect to learning^and 

v 

a particular grade and- instructional a'rea. 

1 
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Sources of grror in*APPLE Observations 

In the main body, of, out report, we discussed sources of error in - 

classroom observation procedures. The primary source of erpor to be 
i 

minimized for most accurate measurement is erroj^JLn^j;he samplipg # of events 
to lie observed. In the APPLE observations this error enters the record: 



■it- 



1) wtven observers dt> riot see an event, 2) when observers position themselves 
by choice, or by circumstances beyond their control, so that observation*^ 
of target pupils is difficult, 3) when observers do not observe the target 
pupils as directed, 4) when there are ^n insufficient number of obser- 
vations of target pupils so that the event record is a biased sample of 
classroom behavior,. 5) when observers do not describe an evdnt with 

enough precision so that it can be encoded unequivocally, and 6) when 
** « 

obserxers^fail^ of .significant events. # , 

, There are other sources of error in the recording of behavior records 
which affect sampling of the classroom behavior, 'but they aire procedural 
rather Jthan sampling errors, and likely "to account for low reliability 
between records 'of observer pairs when they observe the same classroom 
on the same day.. Some of these procedural problems are: .1) not 
establishing times to begin and to end the observation recotd, 2) not 
agr^eing^on a procedure for the order in which/ .-target pupils are to be 
observed, aad 3)" observing in classrooms, where movement around the room 



is impossibly, so that target pupils are observed from different vantage 
points. Procedural problems 1) and 2) carr be minimized with instructions 
abput when to begin and end the .dual observations and about how to decide 
initially the order of observation of target pupils^ Problem 3) can he . 
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minimized by having more cteys of dual observation so that there is a 

' ' ' * * • 

larger event record and consequently a more adequate sampling of class- 
room behavior. \ ' „" I \ 

Types of sampling* errol: in the APPLE observation record s Reliability' 
jof-reports of instructional contexts are subject to sampling error due to: 
i) failure of the observer to record aUL -C^ntext^^ pupils 4 

and 2) the "position of the observer in 'the class-room. The former source 
of error can be minimized by observer training; the latter usually is 
dependent upon the degree to which all of the target pupils\are easily 
observed. For example, in "open" classrooms where pupils wor)^ indepen- 
dently on a variety of activities during an instructional period, it may 
be difficult to observe all of the context changes; In. some BTES class- 
rooms, target pupils were moved back and forth from* an Independent- 
IndividualV to a Teacher-Individual, to an Adult- Individual, and then to an 
Independent-Group context as they finished one assignment, went on to get 
instructions for another, began and finished that one, and then moved on 
to a previously designated activity. When "all target pupils are vajr^ously 
engaged in such movement, error enters the observation record because 
the observers cannot obtain a complete sampling of context changes. More 
observations would redbce sampling error. 

Error in the sampling of teacher activities is 'dependent on error 
in the sample of target pupil behaviors. Sampling erro£' in the record 
of teacher activities occurs when the teaching activity associated with 
the target pupils is not an adequate, sampling of teaching activity for 
the class as a whole. This error can be minimized by having more 
obsen/ations, as well as controls for the above-mentioned sources of- 
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errdr in the -pupil event record. Teacher responses are also dependent 
on the sampling of behavior fpr the .target pupils, and sampling bias is 
the record of teacher responses can be kept minimal by control of the 
sources of error in the pupil event record, \ 

^ Types of procedural error in the APPLE observation system . The 
procedures for observation in the BTES project provided for 20 days of 
observation by pairs of APPLE observers. These days were scheduled by 
the ETS Berkeley office on di£fe^ent dates throughout the observation 
period. There were no special instructions for the APPLE observers on 
the paired] observation days, a factor which introduced procedural error 
over which! we had no control. The observers confronted other problems, 

^ / - 

such as^Eeachers scheduling a movie for the reading period, teachers arranging 
for instruction other than what was expected, teachers being absent, or 
observers being late for the observation. Parenthetically, we^Jould like 
to note that these observations were carried on ift^the mi-dst of the Gasoline 
Crisis of Spring, 1974. Our observers had to drive long distances and 
'getting and having enough gas, as well as arriving for their assignments 
punctually, presented a genuine problem. - , 

In additfon to procedural problems introduced by unexpected class- 
room circumstances and scheduling difficulties attributed to the travel 
situation, there are other types of procedural error which can erfter 
the classroom behavior Record. Procedural, rather than sampling, error 
occurs when observers fail to make a descriptive record, and use * 

inferential statements like "teacher praised the subject, " rather than 

** ' < * 

"teacher said, 'That is a very good answer. 1 " These two records of the' 
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classroom occurrence cause encoding ambiguity. Other encoding difficulties 

which ar6 sources of procedural rather tharr sampling error result, for 

example, from incomplete behavior observations " so that the encoder' ha§ 

to infer the intent of the observer from*the overall content of the 

'classroom record. 

Procedural errorjcan be ^reduced m^isurably by providing a longer 

training period for observers, b^estkblishing methods for conducting — 

\ ' - 

paired observation which will produce recprds of the same time period, by \ 

encoding observations £s soon as possible for the first few observation 

* - • + > 

periods conducted upon completion of the training sessions, and by'indivi-* • 
dualized feedback to an observer relative to the objectivity and completeness, 

of the observation record. ' ' 

• * * 

The Reliability of APPLE Observations for Single Day Samples of Reading 
and Mathematics Instruction , 

The data collected "from the duai observations nevertheless provided 
us with tentative evidence about the reliability of the. APPLE observations, 
as we^l as- detailed information about the sources of sampling and pro- 
cedureal error to which the system is vulnerable. The results of this 
inquiry enabled us to specify more clearly sources of sampling error and 
a :check list, of procedural requirements for reliable observations. 

After all of the observations had been encoded and processed into 

the computer files, we 'retrieved the observation record for the pairs of 

observers and computed the -reliability of the pupil events, contexts, 

teacher activities and teacher responses* The non-normal distribution 

of the items in the observation categories precluded the use of parametric 

data analysis procedures. -We considered other statistical methods, 

* j 

including score transformation, but decided to analyze the data as they j . 
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stood, using a non-parametric method. We selected rank order correlation 
as the statistic which would provide ap estimate of *he degree . to. which 
the observers agreed on tjrrtz frequency of classroom behaviors.^ We computed 
the rank order correlation for each pair of observers for events, contexts 3 
activities and responses. These correlations are reported in Table 86. 

y For each pair of observers, we examined the data to determine which' 
source of sampling or procedural error accounted foj: lowest reliabilities. 
This intensive review of the observer pairs with lowest reliabilities 
produced no evidence for a systematic source of sampling bias or .error 
which would uniformly apply to 'all cases.. 

The -next step was to' evaluate whether low reliabilities were the 
result of s sampling rather than procedural error. We first determined 
whether observer pairs who observed oh days occurring immediately after 
training had lower reliabilities than observer pairs working later in the 
observation schedule. We found no evidence of an experience effect on the 
observation reliability data. ^ ? 

We next returned to the data sources and considered four^aspects of 
the observations which might be implicated in the low reliabilities. 1 . To 
check these four possibilities we: 

1. counted, the total, number of -even/s for the individual 

observer of each observer pair y&y subject, area to assess ' 
sampling error due to insufficient numbers of observations " 
from one or another observer^ 
"2. -counted the number of events observed for each target pupil 
. to assess observer bias in the frequency with which each 
pupil of the set of target pupils was observed; 
0 3. compared tallies of- individual event names for each observer 
pair to assess observer bias in' sampling .classroom behavior; 
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4. inspected source documents^to check for congruence between 

observers in the record of target pupil behavior. 
Low reliabilities for five classrooms were directly the result of 
an^insuf f ic;ient ^ample of behaviors, as reflected in low frequency of 
observations. Since low frequencies in observational records did not 
typify the observers who made these observations, we considered the low 
umber of observations to be a function' of pircumstances or the classroom 
setting. Consequently, we did not consider the' paired-observatitm data 
complete enough to estimate reliability. In one cj^ssroom 5 for example, 
one observer, failed to observe the afternoon reading group. ,The number 
of* his observations was low and the time in j^Mch^the paired observations 
were*made did oot correspond-.- 



Review of the observation records suggests the following possible 



explanations for the low cbservec^^ between 
^soroe observers. (Refje-r^also to Table 86, p. 208.) 

I, Slight variations between observers in proSe style and/or 

%**n^^" K '*'* " . , ^ ^^^^ " ■ — — fr~ 

ition » 



ability to' perceive aad record important versus^trivial informat 

produced differences in the pattern of quality ratings (the "+," "0," • 

or "- , accompanying the event), even though the events themselves were 

* » 

correctly and Reliably recorded; thus the rank o^der correlations are 
low. This was true of tl^e pupil events in reading for classroom H . 
and the pupil events in, math for classroom. I. 4 

2. Variability between observers in ability or willingness to ^ * 

V » 

produce precise detailed non-inferential records of ongoing classroom 

/ ** 
interactions ^precluded accurate encoding and lowered correlations between 

observers even though similar behaviors were recorded. This explanation 

accounts for the low* correlations between observers for math teacher 
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responses in classroom P, for, math teacher activities and responses in 

classroom B, for math pupil events in classroom E, math pupil events 

s 

and activities in classroom K; and for math teacher responses in class- 

. room M. \ . 

. 3„ Variability "between observers in the amount or time spent in 
a partietil^r location (most likely with a particular group) introduced 
sampling error and produced discrepancies, in the frequency with which 
target children were observed and thereby" lowered correlations. "This 
was a likely explanation f or ,the low between-pbserver correlations for 
reading pupil events in classrooms T and S. 

4. Variability between observers in their understanding of the 
applicability of certain lexicon terms for items which were encoded at 
the time of observation and thus not reviewed at t£e time of encoding by 
the project staff (for example, contexts) introduced discrepancies^lrf€o^ 
the observafinon^eco^ proved to be the case f o rt^ac i n g 
1 n s^?ftrTT^nYi l—eoRfee3rts-~iirxl3rs sroom R and the math instructional con- 
texts in classroom S. x 

5. Variability between observers in the total number of events 
documented also influenced reliability. Classrooms J, K, and E for 
reading and _G_ and^F f or^math vere dropped from* the presentation* of data 
because one observer of the pair made too few observations allow 
adequate comparisons between' observers. This happened, np doubt, because 
of discrepancies between arrival times .or times of observations. 
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Discussion of Reliability of the APPLE System - * . ' ' ; 

.The observation ferror reflected in the reported reliabilities \ 
appears to reflect procedural rather tharf ^sampling error. Training *. 
. procedures ^or reliability studies which include standards for beginning 
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and ending the observation, and procedures for the ^fder of "observation 
\fcff target pupils will reduce procedural ejpKfr cpnsiderably . The ddta 
from the BTES project supplies ijnpdrtant information on the likelihood 
of behaviors appearing during reading, and mathematics. In future obser- 

vation training sessions, the most frequently appearing classroom behaviors 

. . * * »> 

can be discussed and contrasted, and the observers can be trained to 

4ifferentiate &mong similar behaviors such as "Engagement, " "Work Habits'* 

and "Academic Performance." Consequently differences in prose style would 

be less consequential in the encoding j£recess . 

Datla regarding relative f reque^n^y-e^pupil and teaehex, behaviors 

provide a basis for determining the number of observ^Tt-ictiis which are 



— necessary in order to insure adequate samplj 



LOT , 



There shouW^^enoughobs^r^TaTions so t*hat the least frequently appearing 

tid observation variable jjoaild- appear at "least once. For example, the 

number - of reading observations for the observer pairs varied from a low 

of 15 in onfe classroom to a high of 123 in another, a result of the extent 

of significant observable teacher and pupil behavior and the lengthy of the 

reading period. For mathematics instruction, the number of observations 
- 

'in-the reliability sub-study varied from 10 to 47 f reflecting the shorter 
periods of time designated for mathematics lessons as well as the lower 
level of pupil actdvity during these instructional periods. 

The median values reported support the reliability^>f the APPLE 
observations. On the basis of the examination of the sources of error 
in the record, we conclude that the differences in frequencies of 
classroom behavior reported by the paired observers are biased due to 
procedural factors primarily, and secondarily to sampling error due to, * v 
insufficient amount of observation time. The first source of error 



* 231 



207 



affects the extent of agreement among observeTsy^feyi^should not affect ' 

the reliability of the data base. The second source <ef error aJLso*aTfeets 

• t. % - 

the extent of agreement among -observers and could have^aru^ff ectTon the 
reliability of the data base if we had had-only'a few teachers in the 
' sample* HoweveV, the sample of teachers at each grade was sufficiently ^ 
large to insure that the observational data can v be considered £ 'reliable 
representation of the classroom behavior in BTES classrooms. 
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APPENDIX B 



.Descriptive Statistics for ^Observation Variables and Time 
Allocation Measures Stratified by Socioeconomic Status of 
Srtiou l Population an d-Geographic Location of School 
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The material in Appendix B contains information on measures of 
central tendency and variability of the APPLE observation variables with 
respect to the socioeconomic status of the school population and the 
* location of £he •schools, in the BTES project. These data illustrate the 
• similarities and differences laraong pupil and teacher behaviors ir\ class- 
rooms .which differ with respe'pt to key demographic factors. This exami- 
nation of the APPLE data relative to stratified groups provides another 

l o ok and a refinement of inf prpnces obtained from t he_exami nation of the 

\ 

total sample., , 

I « > • 

Contrasts Between Prorated Frequencies of Observation Variables for Schools 

Which Differ with Respect to Socioeconomic Status Level 

1 

Pupil events . When the average prorated, incidence of observed pupil 

i . 
events in reading was examined jfor schools with differing socioeconomic 

status (SES) levels, there was more observable evidence of "+ Oral Reading" 

in high SES schools (Tables 87 knd 88). * "+ Number Concepts 11 was most fre- 

quent in high SES schools for* second* grade , but the average incidence of 

I 

this event was similar across aljl schools at fifth grade (Tables 89 and 

! 

90). At second grade, "+ Engagement 11 events were recorded most frequently 

for the middle SES classrooms .inj both reading and mathematics instruction, 

i » 

Regardless of area of instruction., the frequencies of tjie negative behavior 

events, Conduct", Inattention", and Talking",* when aggregated, - 

decreased *w*ith increasing SES levjel at second grade. The SES extremes had 

the highest combined frequencies pf these negative events in fifth grade 
/ 

classrooms . 

Contexts . The "Independent-Group" and "Teacher-Group" contexts were most 
frequent among all SES levels for second grade reading (Table 91). For 
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fifth grade reading » (Table 92) r""Independent-Cl5ss" instruction was most 
common*. in the high SES schools, while "Independent-Group" contexts were more 
frequently reported for low and middle SES classrooms. Instruction of the 
total class, either in .teacher-conducted or independent work sessions was 
observed more frequently in fifth grade high SES classrooms. Classroom 
work independent of teacher direction increased with grade level fpr all 
SES groups. j 

J-C 



"Teacher-Class," Independen 



generally predominated in mathema 



-Group," and "Independent-Class" instruction 



ics lessons (Tables 93 and 94). "Teacher- 
Group" contexts appeared more frequently at the middle than at low or high 
SES schools. Individualized instruction, while relatively rare overall, 
occurred most frequently in second grade classrooms of middle SES schools. 
There w^s relatively little "Adult-Group" instruction in mathematics except 
in low SES second grade classrooms. In general, the differing frequencies 
pf observed instructional context with respect^'to grade level and SES 
probably does reflect real differences in instructional approach in these 
schools. . - • 

Teacher activities . "Working With" was the most frequently occurring 
teacher activity across all, SES, grade, and instructional levels. However, 
teachers in the middle SES classrooms were observed more frequently than 
those in low or high SES classrooms "Working With" groups or individuals in 
reading and mathematics -at second grade (Tables 95 and 97). In fifth 
grade, "Working With" was most frequently observed in low SES classrooms 
in reading and in high SES classrooms in mathematics (Tables 96 and 98) . 
"Circulating" and "Asking" were common teacher activities daring reading 
instruction in low SES classrooms. "Circulating" also occurred with 
considerable frequency |n the middle SES classes, particularly in fifth 
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\ 

grade. "Checking 1 ' appeared frequently only for the high SES groups. The 
"At Desk" acti-vity was\ore frequent generally in the fifth grade than 
second grade, and most frec^uent for high SES groups during reading. v 
In mathematics instruction, x "Circulating" was common at all levels 

except for the middle SES fifth grad^ classes. At second grade level, 

\ 

"Checking" student work was frequent for- middle and high SES groups, but 

was observed with very low frequency for the low SES classes. Both 

"Checking" and "Helping" were frequent across SES levels ^at fifth grade, 

and "At Desk 1 ^ was also frequent f or^the-^w^J>E^<^ 

Teacher responses . Teacher responses were observed to^vary little 

between second and fifth grade classrooms and between reading and mathe^ ~ 

matics. In the examination of possible differences in teacher responses 

with respect to level of schools, the data presented in Tables 99 

through 102 are of interest. For second grade reading, "Positive Feedback" 

and "Praise" were at about the same level across "all SES groups. In reading 

instruction at the fifth grade, "Positive Feedback" and "Praise" were 

higher at the high SES level than for either the middle or low SES groups . 

'"Negative Feedback", in reading, while occurring with rare frequency, 

occurred most in the middle SES classes at both second and fifth grade, 

"Disciplining" responses were most frequent for high SES reading instruc- 

m 

tion at both grade levels, while "Teacher Help" was more characteristic of 

t^ie low SES classes, particularly in second grade* 

"Positive Feedback" was observed to occur least frequently in the 

second grade low SES mathematics lessons. The frequency of "Positive 

Feedback," "Praise" and "Negative Feedback" was consistently higher in the 

high SES groups at this grade level. In fifth grade mathematics instruction 

1 

middle SES had the lowest frequency of either positive or?' negative feedback, 

. 2id ' 
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with some exceptions. M + Attentiveness 11 and pupil events w h^ ^ were 
evidence of "+ Number Concepts 11 were highest in the urban classrooms. 
However, when all of the number concept events were , combined, there were 
frequent occurrences of behaviors associated with evidence of arithmetical 
concepts, positive, negative and neutral, in the rural classrooms. As in 
second grade mathematics classes, negative pupil behaviors were least 
frequent, on the whole, in the fifth grade urban classes. 

Contexts . ,! Teacher-Group ,! contexts were the most common for reading 
instruction in all locations, 'followed by "Independent-Group" contexts. 
There were greater frequencies of pupil behavior in the "Independent-Indi- 
vidual 11 contexts in the urban classrooms at second grade (Table 107)* 
Instruction in groups by an adult other than the teacher was most fre- 
quent, though relatively rare overall, in the suburban second grade 
reading program. 

"Teaeher -Class" and "Independent-Group 11 cotitexts were prominent 
styles of organization for instruction in mathematics at second grade 
(Table 109). The "Teacher-Group" and "Independent-Individual" contexts 
0 were least prevalent in rural areas, suggesting that there might have' been 
f less individualization of mathematics instruction in these schools* The 
relatively h^^jr frequency in rural areas of "Teacher-Class" and "Indepen- 
dent-Class" contexts would tend to support this conclusion* On the other 
hand, "Independent-Individual" and "Independent-Group" work had the highest 
relative frequency in second grade suburban" Mathematics instruction, 
leading to a tentative hypothesis that individualized instruction in 
second grade mathematics was most common in suburban classrooms and least 
common in schools located in rural areas. 
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In" fifth grade, with the exception of urban areas, learning activi- 
ties were carried on independently of direct teacher involvement in over 
fifty percent of the observations (Tables 108, and 110). While almost half 
of the instruction in the urban fifth grade mathematics classes was "Inde- 
pendent-Class" or "Independent-Grpup , M "Teacher-Class" and "Teacher-Group 11 
instruction characterized the remainder of the observations at a much 
higher level than all teacher-directed learning in mathematics in suburban 
or rural areas. Evidence of individualized assignments, while infrequent, 
generally were highest in the schools located in the suburban areas for 
both reading and mathematics, * 

Teacher activities . The tables summarizing teacher activities (Tables 

> 

111 'through- 114) provide evidence about consistency of teacher behavior 
across geographic location, grade and instructional area. In comparison' , 
with other activities, "Working With"* was observed most frequently in all 
groups. However teachers in urban classrooms always had relatively 
higher prorated frequencies of "Working With" activities , than teachers in 
suburban and rural classrooms. These urban f^achers were next most fre- 
quently observed in "Asking"' activities in reading, and in "Circulating" 
and being "At pesk" or "At Board" in mathematics. Except for second grade 
reading instruction in rural a^J|as, suburban and rural' teachers were 
observed in next orde'r of frequency to be "Circulating" around 'the room or 
"Checking 11 student work. In reading instruction .[at second "and, fifth 
.grade, carrying on "Discussion" sessions were more common in the rural 
classrooms than the urban or suburban. t 

Teacher responses . Teacher responses to^upil events had generally 

similar levels, of occurrence in all geographic locations (Tables 115 thjroi/gh 

• *> 

118). . It is Qf interest, however, to examine ^he frequencies of "Positive 
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Feedback" and "Negative Feedback" as well as instances of "Disciplining* 1 . 
and "Praise" in order to locate_*lif f erences in teacher responses which 

might be associated with the locale, of the classrooms* In reading instruc- 

\ 

tion, "Positive Feedback" was more ^frequent in the rural setting. When 
"praise" ^nd "Positive-Feedback" were Combined, the total frequencies 
occurring in reading instruction were a\t similar levels In all locations. 
In mathematics, the combined rates of "Positive Feedback" and "Praise" were 
lowest fur the rural classed, and highest for suburban at second grade and 
for urban at fifth grade. In all settings "Negative Feedback 11 wag uncommon 
and occurred with similar frequency in both instructional areas, at bath - _ 
grade levels and in all locations. Evidence of "Disciplining" was highest 
in second grade suburban classes in reading and lowest iit these same 
classes in mathematics. When the rates of disciplining were combined ovevjy^ 
all instructional areak and grade Xevels, there ^eemed to be no difference 

\ * . / : 

on the whole in the instances of pupil behavior requiring disciplinary 
action which could t?e associated with the urbanization of the schools. 

Comparisons of Time Allocation in Instructional Contexts for Schools Differing 

- ? 

/ 

with Respect to Socioeconomic/ Status Level 

— 

The" distribution o|/number of minutes in each context, the number of 

/ 

productive, minutes^ind the percent of overall time and. productive time in 
contexts, stratified by SES level for second grade reading, are presented 
,i*n Tablj^^ll9 through 12^. Each table provides the* mean, median and range 
^fctfr each of the contexts. When data are presented for the percent of 
productive time in context and the percent of gross productive time, only 
those classrooms in which pupils worked in that context were^ included in 
the calculations. For these tables, we indicated the actual number of 
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classrooms iu which pupils were observed in each .context (such as Tables 
122 and 124). The data presented in the remainder erf/ the tables include 
al-1 of the classrooms in the computation of the means and medians and the 
percen't of time in contexts. (Refer to pp. 59 - 64 in the body of this 
report for a complete description of these timfe allocation variables-.) 

As the reader scans these tables for f ings of interest , there will 

1>e several ways in which the review can be approached. One can examine 
the contexts in whichmost, time was spent across grade levels or across 
instruction within a grade level. Then by comparing the extent of produc- 
tivity in context one has a basis for hypothesizing the ?relationship 
between context and productivity by grad£*and instructional are'a. Another 
way of reviewing ~ t+*e j^bles* might "be to examine- differences. wi thin a j 
particular context. For example, infwhat a*reas ,_ and at what-^rade. are the 

.individual contexts most , cbrainon? What proportion of classrooms utilize 
different types of individual" instruction? . How productive are pupils in 
these contexts? In the discussion: of these results, w6 will irjake^some 
. generalizations about possible important differences 'In instruction which 
reflect the ways in which teachers organise learning experiences in ^schools 
with populations from differenc socioeconomic status levels. In^a later 

J -> 

section we will examine differences with {respect to loc^tion_af-^ife^ school. 
At / the 'second grade, across all SES ^le^hsrf~~mov e pupil, time j^a-s^spent 




in Independent-Grou 
in order of amount of time spent, 
pointed out earlier that there" seemed to 



contexts . Second 
Teacher-Group context. We have 
be evidence tjiat the fiddle SES, 



classrooms were the ones which employed the Independent-Individual contexts 
to the greatest extent, ^he time* allocation data support "this conclusion 
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since the context which averages third highest in time for the middle SES 
schools was Independent-Individual. In the high and low SES classrooms, 
Teacher-Class contexts ranked third. 

At least 50 percent o£ the reading instruction time of second graders 
was speat in Teacher-Group and Independent-Group contexts^. ^The differences 

between the SES levels of the schools were most prominent in the extent of 

~~ 1 * * ►* 

t 

Teacher-Class instruction.. In the low SES schools, 24 percent oi; the tifliie 
was in this context, while it occurred only six percent of the time in 
middle SES schools. At the low SES levels, there was a greater extent of 

'reading instruction by class (T-Class and I-Class) than at the other two 
socioeconomic status levels. In th^ high SES schools, the proportion of 
individual instruction was less than for middle and low SES classrooms. 

"The extent of fifth grade reading instruction by class ( 1 T ::: Ciass and 
I^Class) in the low SES classrooms was "generally the same as at^second 
grade.* However, there were large differences between grade levels in the 

\ extent to which instruction was by the teacher to the class as a unit in 
' , the high SES schools (Tables 125 - 130). Table 127 shoJs that 63 percent 
of reading instruction time in high SES classrooms was in T-Class and I- 
Class contexts whil'e only 34 percent 'of time in middle SES Alassrooms and 

'•' V 

27"»percent of time in low SES classrooms was in these contexts. The 

'm * 

'proportion of time in instruction by group (T-Group and I -Group) remained 

* " the sajne in the fift^h grade low SES classes and middle SES classes as alt 

" ' ' * J 

was i'rt second grade,' but it decreased to 30 percent (cpmparedf with 74 per- 

cent 'aff second *grade) in the high SES classes. teacher-Class 'instruction 
I ' J. * 

was judged to be. .the most productive 'of air context? in fifth grade 
reading for the hi'gh^SES classes arid the low SES clas.ses although' the 
percent in the latjter* case was considerately lower. Teacher-Group instruc- 

ERIC . ' . : : • 
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tian, in reading in th<? middle SES schools was a relatively frequent 
occurrence and judged to be a highly productive setting^for pupils. ' 

This examination- of cime-allocation in contexts, in reading Shows that 
the distribution of time was, fairly consistent between grades in the low. 
SES schools, but changed markedly between second and- fifth grade in .the 



hit;h SES classrooms. The data alsc/show that individualized instruction 
occurred most frequently at the middle SES schools ^ ranking third after 
group instruction ' in time allocation. 

c . I 

O + # 

The Eables ndxt in oi;der are^feh^se^reporting^the time allocation data 

tor second grade (Tables^! 31 t>*dugh 13'6) and fifth grade • (Tables 137. 

throu-ft*r~T2f2) mathemj^rtfs. Second grade pupils in the high SES schools 

were i w^Hrfed in instruction bj/. the teacher^to the entire class 75 percent 

of the time, while group instruction occupied 21 percent of the total,, 

number of minutes reported for. mathematics in these classes. Pour contexts 

dominated ' Mathematics instruction &t both grade ^levfels,* but there were ' 

.differences among &ES level of sch^l i,n the extent ' ' ojL-groxxp instruction. 

In the. middle SES schools thel* time. spetiT'in^t^ by cla's'S and by, 

£rouo In mathematics ,was similar. In the low and high Sp{5 ' schools;loonsider 

ably more time was^pent in instruction by "class. Independent,- Individual - 

contexts were a relatiyel^ rare form, of mathematics instruc tion ' at second 

gr^de involving only a few low and middle SES ,cla"ssr,oomsf and only two 

classrooms at fifth grade, (both low SES), '* 

*The data characterize mathematics instruction at. fifth grade^as being 
* i t> ** 

carried otft, in contexts where the ciass works independently qf/'the teacher 

' . * , > ' \ • il K • " m .* . 

either as a total unit On the same assignment ' Qr 'ip groups with assignments 

according to learning level- For -example, even though much time was 1 

J r # 

devoted to Che. T^acher.-Clas& context in the low" 'SES glasses , the combined 
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time spenfin the Independent-Class and Indeperident-Group contexts in°the 
low SES classrooms was greater than tKd "combined contexts with teachers 
directly in charge of the instruction. ' t f 

When one examines the second grade mathematics tables for evidence of 
the contexts in which thehre were greater levels or productivity as'judged 
b/the observers, both class and group instruction had higli levels of 
productivity in the highland middle SES classes. The^ productivity level 
was lower in the low SES classes with the exception of the Teacher-Glass * 
context. It is ~of interest that of all SES levels, Teacher-Group instruc- 
tion was observed to^be least productive* for the low SES pupils. In the 
low SES classes the relatively low le-fcel of observed productive worK and 

the indication of a rel ianc e "on" ins truc^ion by_clas*T or independently^ of_ — 

* 

the teacher raises the question of the degree of tfeacher dependence- oxj 
curriculum materials or textbooks and their effectiveness for guiding the 
learning experiences of children in mathematics. 

Mathematics iT*FE*w^i° n of Athe c l ass as ? unit (Independent-Class) 
was most common in the fifth grade high and middle SES schools* Teacher- 
Class instruction was still prevalent in mathematics lessons for the low 
SES classes, but an almost identical proportion of time was spent in the 
Independent-Class context. When the time all^ation units associated with 
instruction by class were examined, the proportions for class instruction 
were similar for secortd and fifth grade low and high SES classrooms, but 
Tables 133 and 139 show that* the percentage of Teacher-Class time at fifth 
grade was considerably than at second grade ej^£^hough the amount of 

r time spei^t in instruction by^class (Teacher-Class time and Independent 
Class time) was similar at both grade levels. This means that as pupils 
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get older a greater proportion of their time ii) mathematics will be guided 
by textbook or workbook learning experiences and les* tiTie will be spent^ 
in teacher-directed activities. ^ 

In comparison to tfa^ other SES levels, the middle SES classrooms had 
the greatest proportion of Teacher-Group and Independent-Group instruction 
, at both fifth and second grade. TJje evidence for more attention to'indivi- 
duai differences in instruction continues to favor the middle SES school^ 
in mathematics as well as in' reading. 

An overview of the success of instruction in these different contexts 
isl available from the observer reports of the levels of target pupil 
productivity in each context. In cfom^aring SES levels at the fifth grade, 
.pupils in high SES schools had the highest levels of productivity and 
pupils in low SES class.es had the lowest levels or productivity in contexts 
with the classroom as the instructional unit.* The percent of gross produc- 
t,ive tim£ (Table 136) for second gr£de pupils in the Teacher-Group context 
was uniformly high across SES levels. While the level of productivity was 
judged high in Individual contexts and in contexts where an adult other 
than the teacher* directed the^ instruction, these contexts occurred infrequently, 

Comparisons of Time Allocation in Instructional Contexts in Reading and 
Mathematics for Urban, Suburban, and "Rural Schools 

In thi$ discussion, the data are comparable to those summarizing time 
* * 

allocation in context for schools stratified by SES level. As in the pre- 
ceeding section, the data for^ each context include the mean, medidt] and 
range of time. Time is reported for each context in terms of the number 
.X>f minutes, the number £>f productive minutes, the mean percent total 
minutes, the mean percent productive minutes, -the mean percent gross time, 
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and the mean percent gross productive time. As before, the computation' of 

proportion of time which was .productive was based only on classrooms in 

J 

which time was allocated in a particular context. When teachers did' not 

^ i 

use a context, time data from their classrooms were not included in the 
productivity computations. 

Tables 143 through 154 describe the time -allocation variables for 
reading instruction. With the exception of rural fifth grade classrooms, 
time spent in Independent-Group contexts in reading ranked highest acrosp* 




ERIC 



grade level and location (Tables 143 and 149) • Independent-Group wqj 

■ / 

in reading is probably accompanied by some time spent in Teacfier-Group 
instruction. If reading instruction is actually or^^nlzed by group, the 
second^r^nked time in context should then be £h€Teacher-Group setting. 
This was the case in reading for all the classrooms at second grade and 
true also for the urban fifth gra$te; A different pattern occurred for 
the suburban and rural* fif tVgrade. In the suburban stratification, 
Independent-Group^ an^J^Teacher-Class contexts ranked first' and second in 
mean time allocation, and in rura^fjr Independent-Class and Independent- 
Group^^re in first and second or^er^ The extent of organization or 
reading instruction by groups was reflected also by^the total percent of 
time spent in the two group contexts (Tables 145 and 151) . Teacher- 
Gtoup and Independent-Group instruction occupied two-thirds of the time 
in the second and fifth grade urban classrooms, about 60 percent of the 

\ 

time in second grade suburban and rural classrooms, and only about 40' 
percent of the time in the fifth grade suburban v and rural schools. The 
largest proportion of time spent in instruction of the class. as a unit was 
in the fi§th grade suburban and rural classrooms. 

For thesfe two groups, from 43 to 53 percent df class time was devoted 
to instruction in which pupils were working with the class as a unit where 
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the assignment and expectancies were presumably the same for all class 
members. The, uniformity of context which we £re inferring from these <Jata 
Ec5r suburban^ fifth graders needs to be contrasted with the fact that:, on ( 
the average, 12 percent of their class time in reading was spent in Ihde- 
perideril>Jndividual contexts — a higher proportion of time than was evident 

in the "urban and rural fifth gra(de (classrooms. ^ 

^ ■ / • . . - " . 1 

The reader will recall* that the incidence, of individual contexts 

^ (J V t 

reported' in conjunction with pupil events was higher at the second grade 
suburban level. The time data Confirm this finding in that the second 
grade suburban classrooms had the highest percent of time in individual 
contexts by schools location.. In addition more time was allocated to in- 
struct ion by teacher-aides as the Adult-Group context information shows. 

When instruction was administered by the teacher, as in the T-Class 
and T-Group contexts, the level of productivity as expressed by the percent 
of gross productive time (Tables 148 and 154) was higher regardless of^ 
location and grade level. Productivity levels in second grade reading 
were relatively uniform across contexts, but productivity was lower in the 
independent contexts, especially for the urban and rural classrooms. The 
poorest levels of observed productivity in fifth grade were again the 
Independent -Class and Independent-Group contexts even though these contexts 
accounted for a large proportion of time allocation in reading*^ 

When one examines the tables reporting^ the time data (Tables 155 
through 166), it is apparent that the schools vary^considerabXy by location 
„ # with respect to the organization of instruction mathematics. IrT^seconcL 
grade, Teacher-Class instruction occupied more time than atvy^Vther instruc- 
tional context in all locations. Urban classrooms at fifth grade were 
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organized overall to distribute roughly equivalent amounts of time^o the 
^several contexts, but Teacher^Class instruction again predominated. In 
the suburban and rural clasSrtfoms at fifth grade, the Independent-Class 
context was most common, followed by Teacher-Class instruction:. In » rural 
classrooms at the second grade level, 8f> percent of the instructional time 

in k mathei]iatics was spent in Teacher-Class and Independent-Class instruc-^ 

' v ' ■ ' - * * * 

tion (Table ^157) Suburban second ^grade classrooms ranked lowest with 

respect to amount of n time devoted to instruction by class, but fifth grade 
suburban classrooms spentmore time than either "urban or rural ^ifth grade 
classes in these contexts. It is of sbme interest that time in the sub-^ 
urban classes at second grade was divided almost equally between grOup_ and' 
class instruction, but that in these same schools at fifth grade, only a 
small portion of mathematics time was devoted tg group Instruction (Tables 
155- and 161). ' ■ ' * ; 

"Ttekhematics instruction in the rural schools was carried on in indivi- 
dual contexts in only .a^few classes. Only three classes from urban and ^ 
suburban settings reported time^r-^udividual contexts and a similarly 
small number of classes had instruction by an adult other' than the teacher. 
These data 4long with those reported previously permit the inference 

> 4 

thajt only a minimal amount of instruction in mathematics can be considered 

individualized, at least based on these time allocation d^ta. While one 

* 

might expect that individual inistruction would produce mote interest on 
the part of the pupils and consequently greater evidence of productivity, 
the time data do not support such an^assu^ption (Tables^l58, 160, 164 
and^^66) . Productivity was very often -as gre^t or -greater in Teacher- 

% 

Class instruction regardless of grade level or location. 
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In general, these statistics contrasting average incidences of 
observation variables and time allocation measures in classrooms strati- 
fied by SES and location do show that demographic variables are related to 
differences in classroom performance. Where no SES or location differences 
exist among the observation variables, we can conclude that future research 
on teacher performance variables utilizing observation variables as mea- 
sures would not have to take into account independent factors stfch as^the 
SES level of the school population or whether the school was located in a 
rural, suburban or uiy^an area. ' * 1 
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IX C 



Factor Analysis and Factor Validity Studies 
for APPLE Observation Variables 
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In Appendix C, results of factor analyses of 4jPPLE variables for 
three major categories of the observation system are reported. Following 
the results of these analyses is presented an examination of the rela- 
tionships of the factor scores derived from the analyses to pupil putcome 
measures in reading*, mathematics and attitude. 

*■ 

Factor Analysis of Observational Variables 

In an effort to tease out any empirical relationships among groups of 
variables within an observational category, factor analyses were conducted 
of the pupil events, contexts, teacher activities and teacher responses by 
grade and instructional area. We recognized that a factor analysis of the 
observation variables isolated from one another into categories rather than 
analyzed together as ^interaction units- .(such as teacher activity-pupil event 



combinations or pupil event-teacher response combinations) would provide an 

* ■ 

incomplete picture of classroom happenings. We conducted the analyses with 
this knowledge in mind, in order^t^ identify particular aspects of an ob- 
servational record which would support the generation of hypotheses* about * 

9 observable pupil or teacher characteristics associated with end of year 
outcbmes. Utilization of factor scores derived from the analysis was depen- 
dent on the interpretability of each of the factor dimensions. If the 
va^i^les gtiigh defined each dimension could be understood to reflect 

-behavioral, management, teaching or (response styles, we accepted them for 
further analysis. • \ . 

c * , .4 • *•'.-.. " 

The 'factpir 'analysis of pupil events produced dimensions which were 

, - .difficult to interpret. However, the factor analyses of contexts, teacher 

1 * * * * * 

activities ^ and -teacher responses provided dimensions which appeared' to have. 

substantial psychological meaning. Our rationale -for the APPLE Observation 
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System is based on the belief that citations of pupil behavior in the 
absence of descriptions of the setting or the instructional behavior of the 
teacher have limited meaning* were not surprised, therefore, that the 
factor dimensions for the pupil behavior events were difficult to interpret, > 
believing as we do tHat the common factors accounting for the interrelation- 
ship, among the pupil variables might be particular instructional contexts 
or teacher activities. For example,' we found that "Direction Following" was 
positively related to incidences of "Attentiveness" in one dimension, ^md 
negatively related to incidences of "Attentiveness" in another dimension. 
Our notion is that the reason^f orchis apparently conflicting result is that 
pupils can be both attentive and follow ejections in contexts where teachers 
are closely in charge of the instruction, while the behavior of .these same 
puj>ils in contexts where thejTarfe working independently of the teacher may 
-evidence ina^ter*§i^>&-even- thou^h^ 3. 



" future time we-piropQse to conduct" factor analyses of pupil behavior .in 

« * - 

conjunction with contexts, activities and responses in order to define 

meaningful classroom interaction units based on naturally appearing teacher- 

< 

pupil behaviors. 

♦ y ~ . * \ 

The rea4er. is correct in-fwondeiriiig whether we hkve a' sufficient ration- 

ale for using Contexts, Teacher Activities and Teacher Responses as isolated 

V 

observation variables in factor analyses when we Jiave considered the results 

of fafctor analyses 'of pupil variables 'to be incomplete when> studied, in 

isolation of teaching behavior. The^f ollowing report of the factor -analyses 

of these observational characteristics of Reaching rest on the assumption, 

~ that the way teachers organize their classroom learning experiences ' 
/ - ' . ' 

(contexts), tfceip instructional behavior (activities) and their responses \ 
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(teacher tesponses) represent stylistic differences between teachers which 
may be relatively independent of the characteristics of their pupils. The 
appropriateness of such^an assumption must be J^gted. For the hypothesis- 
generating objective of the Phase II BTES study, we believed that we did 
have "sufficient basis to assume that we could analyze teacher behavior 
independently of pupil behavior. 

Instructional contexts * The results of the factor analyses for feading" 
instructional contexts for second grade and for k fifth grade, clustered the 
variables in different ways. At the second-grade -level (Table 167) the* di- 
mensions were Total Classroom Instruction, Group Instruction and Indiyidu- 

— • %. 

alized Instruction, a factor which reflects the extent to which children in 

«- i,. 

the classrooms work independently of the teacher on assignments which are 
different from those of* their classmates. Each of the two dimensions for 
contexts at fifth grade (Table 168), Independent Woi?k by Class and teacher 



Instruction of Class were defined by -a negative loading on group instruction 
variables. fhe first* factor, Teacher-Group contexts, \>had ^a. negative coeffi- 

cient while the coefficient fox; the Independent-Clas,s contexts was -positive. 

* & i. 

Independent-Group contexts had a negative loading and Teacher-Class context 

l. 

had a positive loading on the ££cond factor. The factor dimensions 'from . 
the analysis of instructional 'contexts for mathematics we;re identical at 

* - s . 

both grades (Tables 169 ^and 170). They were Teacher Instruction of Cla 



SS, 



Independent Work by Class and. Group Instruction. 

Teacher activities . The factor analysis of ^teacher activities yielded". 

a ■ 

multiple dimensions, differing between. grade levels and inst ruetional areas. 
Teacher activities associated with introducing a lesson were, however, common 
to all classroom groups (Tables 17*1 through 174). ""Circulating" around the' 



Table 167 



Factor Pattern Matrix for Instructional Contexts 
in Reading — Grade .2 









* 

Factors 




Contexts 


Total 
Classroom 
Instruction 


(?roup 
Instruction 


Individualized 

9 

Instruction 


Ar^Group 


" '-.09 


.49* 
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-.06 w , 


/I- - Ciass\ ' 

;t • . 
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f. * V 
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I Individual 
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Table 168 

Factor Pattern Matrix for Instructional Contexts 

* * 

in Reading — Grade 5 







Factors 








X t CIA*. 1 1C u 






•Contexts 


, Independent 
Work by Class 


Instruction 
of Class 
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- Grqup 


' : -.16 
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- Class 
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- Group 
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- Individual 
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Table 169 

Factor Pattern Matrix &or Instructional Contexts 

» 

in Mathematics — Grade 2 



— — r 


- 


Factors " 


* ■* 


0 s 

Contexts 


Teacher 
Instruction 
of Class 


Independent 
Work by Class 


Gr.oup 
Instruction 


A 7 tJroup 


-.21 
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t.33* , 


I - Class 
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-.07 , 
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I - Individual 

f 
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T - Class 
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* I ■ - ■ f 
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Table 170 . * 
Factor PaORfir Matrix for Instructional Contexts 
in/MatheWtics Grade 5" 







Factors 
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jroora was the' teacher] activity with highest loading on a*second factor 

* - ; ' 

(called Circulating) which was common to second reading instruction , (Table 

171) and reading and caathematics lessons at fifth grade. (Tables 172 and 

1,74). A dimension tfhich ran be generica^ly described Discussion was 

also defined for second grade reading and fifth grade reading and .'math'e- 

■ ' . \ 

matics. A similar dimension which can be called ^king w ^s present in' 
the, analysis of activities for second and 1 fifth grade" mathematics . Yn 

reading* lesfsons at second and^, fifth grade there was a dimension jwhich we » 

■ . • _ ■ " i * > . \ * - 

called Nunin-ieractive Supervision, Teacher .Directed Instruction and Checking 
were two other 'dimensions for second grafte teacher activities/ Helping, , 
Instructing- by Drill and ^Reyiew, and Available for Help were the, additional 

factors for second grade mathematics. ^Instructing^Groups or Individuals 

* ~ f 1 * /. * 

was a dimension common to fifth grade r.eading and mathematics. Giving „ *, 

* • - • j 

Instruction at Board was a' final factor for fifth grade reading, and At 

- r , 

Desk and Checking Pupil Work exhausted the dimensions of teacher activities* 

associated with fifth grade mathematics,* -\ '* 

In summary, ttte factors defining teacher activities might be described 
*" ' * . * 

as reflecting organizing activities which introduce the. lesson, ins trufct ions 

about 'the lesson, circulating around the room while„.pupils work, npn-inter*- 

active supervision, and^_s^^^ types of instructional work with * 

pupils depending on tha ^rade level and instructional area. 

Teacher responses . Four dimensions of teacher response occurred with 

considerable regularity across grade level and type of instruction (Table ' , 

175^ through 178). A tendency to give No Response to pupil behaviot was a 

• \ 

dimension in the second grade reading* and mathematics lessons and in fifth 
grade mathematics. A Pusitive Reinforcement Responses, dimension was common 
to all, except fifth grade mathematics. 'A Negative Reinforcement Responses 

343 ' : 
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( 

dimension occurred for second grade mathematics and fifth grade reading and 
mathematics, while a Neutral Feedback Responses dimension appeared in 
second and fifth grade reading. Additional" teacher j£spons v e dimensions 
at second ^rade were Instruct ional Feedback Responses in reading, and 
* Ignoring Response's and Questioning Responses in mathematics. There was* 
an Instructional Helping Responses factor in 'fifth grade reading and 
three additional management dimensions in fifth grade mathematics—Redi- ^ 
rectin& x Responses, Verbal Reinforcement Responses and *Non r verbal Acknow- 
led gement Responses.. We assigned labels to the teacher response dimensions 

on the "basis of\the variables which were most salient to each and" attempted 

\\ ^ 

to find terms whl^^w^re common to as many of the grade level and iastruc- 

tional programs as possib] 

* 

Partial Correlations of Factor Scores from the Analysis spf Instructional 
C ontext , Teacher Activities and Teacher Responses with Reading Outcome Measures 

The data presented in the main body of this report contrasted grade 

i * » 

lev^l differences in the relationship of the observational variables to , 

outcome measures. These findings lead to the expectation that the magnitudes 

* and directions of correlations of the observation variable factor scores 

'with outcome measures might also differ with respect to grade. 

i Instructional Context . Such differences were evident in the data 

v* 

presented in Table 179, which showed that the factor scores representing 

instruction by class* were positively correlated with the decoding test^at 

second .grade level and^hegatively correlated at fif Upgrade level. The 

* 

fifth grade Independent Work, by Class factor was significantly correlated 

r 

with the reading achievement outcome. This. implies that reading achievement 

is facilitated when pupils work independently of the teacher. ^Factor scores 

/ . m ' 

of the incidence of different types of contexts reflect types of instruction- 
al organization. Few of these factor measures are correlated with reading 
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achievement outcomes, suggesting that instructional organisation per _se 
is not necessarily a critical variable in predicting reading achievement. 
Teacher activities . Among the teacher activity factors (Ta.ble 180) >. 
. it is of interest that the factor labeled Teacher-directed Instruction 
was positively correlated with the^decocjing test, but negatively corre- 

s 

lated with the California Achievement Test. This finding, and the fact that 
z_ the Total Classroom Instruction conpext dimension was also positively 
correlated with the decoding test, provides conf irmatory .evidence that 
the extent to which teachers direct learning activities in second grade 
reading is related to pupil gain in decoding skills, but not to other 
types of reading achievement* Decoding skills may be learned best at 
this gn^de level from teachers who direct the learning experiences, 
while reading comprehensions skills may best be acquired in classrooms 
where teachers provide more opportunities for interacting with the 

lesson material. This explanation is further supported by the fact that 

*■ c 
the extent of Non-interactive Supervision is negatively correlated with - 

decoding at second grade and that Discussing and Giving Instructions was 

positively correlated with the -Calif ornia Achievement Test. 9 

^t fifth grade level the Giving Instructions at Board factor has a 

negative relationship to decoding skills and Circulating was negative^ , 

correlated with the California Achievement Test. More interactive or neu- 

tr;al involvement o^f teachers as reflected in the Discussing and Explaining 

Lesson or the Non-interac tiye Supervision factors appear to promote the . 

' general reading skills of fifth graders as the data in Table 180 indicate.^ 
Teacher responses . The differing relationships between teacher 
behavior and reading achievement at second and fifth grade levels are 

. well-illustrated by the correlations of teacher response factor scores with 
the reading achievement outcomes measures (Table 181) „ The Positive 
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Reinforcement factor was positively , related to all of the second grade 
reading achievement measures with the exception of decoding. The instruc- 
tional Feedback Responses factor was also uniform in its positive rela- 
tionship with reading outcomes at second grade. The only type of teacher 
response factor associated with improved decoding achievement was Strong 
Positive Reinforcement^ Responses. The most salient of tha variables in 
this factor was the teacher response "Prais'e". 

At the fifth grade level, only one of the Feedback factor scores appear 
to be related to reading achievement. Although the correlation was negative, 
the Instructional Helping Responses factor 'was significantly related to the . 
CAT reacjirfg md&uye. This result seems to imply that teachers who need to 

" provide ,belp invading are teachers whose lessons are too difficult or 
« « 

whose pupils need some type of instruction^JLa^^vi-t^ other than -the one 
requiring teacher help. , * ^ 

Partial Correlations of Factor Scores from the Analysis of Instructional 
Context) Teacher Activitjes^and Teacher Responses with ^Mathematics Outcome " 
Measures • • 

* 

Instructional Contexts . The data has consistently shown that a. set 
of teacher performance variables will not have the same relationship across 
instructional areas and grade levels, ^he differing relationships between 
context factors and mathematics achievement outcomes uphold this pattern 
(Table 1&2) * The Independent Work by Class context factor was positively* 
correlated with the second grade total mathematics outcome measure, but 
correlated negatively, ^though not significantly, with the mathematics 
achievement measures'at fifth grade. Teacher Instruction of Class and Group 
Instruction factors had significant relationships to mathematics ^chievement 
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outcomes at fifth grade, but to none of the mathematic^feutcomes at, second. 

i Teacher activities . Reference to Table 183, which presents the 
"correlations of teacher activity factor scores with mathematics, achievement 



outcomes," shows that few of these factors correlated with mathematics 
achievement scores at second grade. The two factor scores with significant 
findings were both correlated with the California Achievement Test Math 

Concept^. Being Available for Help to pupils as they needed it was posi- 

\ 

tively correlated with mathematics concepts, while the Helping Interactions 
with Pupils factor ^had a negative relationship with it. 

Fivelof the seven teacher activity factors for fifth grade had signi- 



ficant relationships to mathematics outcomes. Asking Questions about 

1 - . . 

Problems, Introducing and Anfewer^ng Questions about Lessons, Instructing 
Groups or Individuals, and Circulating all had significant positive corre- 
lations witfy one or more mathematics achievement measures. The At Dpsk 1 
factor was njpgatiyely correlated with the mathematics applicationy{est . 
These findings lead to the hypothesis that the more fifth grad/ teachers 
4 interact with pupils in mathematics instruction, the better/£heir achieve- 
ment outcome's! will be. 

Teacher' r,espo&ses . It seems of particular interest that an almost 
reverse' patterh of the effedt of teacher, respond on pupil outcomes "occ 
between grade ]ieve Is and between reading^ritfmathematics instruction'. Table 
181 illustrated! the finding that teacher feedback resggjae^ of a variety of 
typfes were positively related^te^resmon^^ outcomes at 

grade, while these responses had no relationship to^ reading achievement at 



, fifth grade. 

Table 184 shows teacher responses in mathematics activities were not 
related to second grade achievement. On the other hand, the Verbal Reinforce 
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ment' factor was positively reLated to fifth, grade mathematics outcomes/ * 
Negative or neutral feedback, as 'rfefilecfted in ttye; Negative .Reinforcement , 

' / / • s - \« 

No*Response, Redirect ing' Responses and Non-verbal Acknowledgement^ factors , 
X ^was negatively correlated with, fifth gr^tfe. mathematics ©becomes.. The 
k only exception was a positive correlation between s the -Negative Reiiiforp^- , * 

« - ■ * v- 

3 * . 

ment factor- and the, Mathematics Application test, * *' ^ * 

We'.cannbt "of fer a, definitive explanation' fx>r^ these differing findings 
with respftt to the observational factor Score -variables . It 'can be^asserted 
however, that jstudies which attempt, to define, the teacher perfornfence 

.characteristics which make a, difcf eren*;e in pup il> achievement will need, to 

\ : , . . * 

\ " x • , • # ' 

take into .account the 'intervening variables of, the age of the pupils being 

c * \ • . . * . < 

» . / v - • " * 

taught and .the type t>f achievement objective. Teacher 'performance oh^rac- 

terristics which affect achievement . outcomes may, or may not*, be .critical 

for positive changes in"' pupil attitudes about school or self. Of interest . 

in this regard is the extent to which performance characteristics' reflected 

• <- • * • 

in these factor scores redated to attitude outcomes. . • 4 ^ 



Partial Correlations of Factor Scores from the Analysis of Instructional 

— — : ; ; ^X-^ : r~. 

Context, Teacher Activities and Teacher Responses with' Attitude Outcomes 

* * • * 

In a, review of the relationship between the factor score data and the v. 

• * • ' * " 0 * 

attitude outcomes, it is important to keep ini mind that- the .attitude ,mea- • 
sures^ as weil as all of the other measures^that we nave reported, are^ 
base^d on the cjasteroora as the unit . of ' analysis . The partial correlations 
reflect the relationship between factor scores for variables based on 
•observations, tallied acrqss*all target pupils and the clas'srtDom aveVages 
'(base^ ort "all pupils) oT^th^attitude and survey , measures controlling for 

Fall 'scores on these tester* In this discussion of the attitude outcomes, 

' - , 3.58 \ * 

ERLC . • % 




the/teader should keep 'in mind that the. factoj score *da.ta represent* ~ • 

CI < - • - > . *. . 

w^b is happening to the class as a* group rather thattf.what is- occurring 

with individual pupils. ► • % • , ■ */ ' 

• . ^Contexts . ^ Tab 1b 185- re.ports 'the partial correlations of trie content 

• • ' 1 * : 

factor sciores with the total attitude'. and t'otal purvey 'measures. None of 
.the context factor scores significantly correlated with .tha fifth grade 
attitude meas.ures.. At §econd grade,' we have* the interesting finding 

• * * S t • \. 

that" all o"f the factor score^3iad .a negative » correlation wj^th.one or. 

more -attitude outcomes. The 'reading factor, Groflp 'Instruction, }was * " , 

.significantly neg*a£i'vely Gdrreldted* with botfr the attitude (positlv^' 

attitude toward reading and mathematics activities) and survey "{positive 

attitude -toward' self ) measures., . The £robp. Instruction factor in mathematics 

lessons was also 'negatively/ related to the second grade survey .measure. 

The conclusion which we can'draw* from these , findings is that the mtfre * 

, ■» * * • 

.opportunity, there is fox, pupils to. .demonstrate their achievement before the 

teacher axid the group, the greater the chance ~that second grade pupils 

will view theLactivity and ' themselves .in a more negative,, light. This 

h^pothesjs was., supported to some extent' by the^sitive correlation of ° 

the "Teacher: Instruction ot Class mathematics factor with the survey, measure 

suggesting , the possibility that the^e is perhaps less risk of self-. 

disclosure in a class instructional setting in mathematics thai) there is . \ 

in 'the* g*oup setting, or than there is in a class, instructional setting^ 
.' ^ # * 

%' 

in^ reading. * • , 6 

• •% 

Teacher 'activities . The hypothesis that poor ajttitude outcomes ar.e 
associated with teacher involvement with pupils in ways which might ?ingle 
them out, is suppor ted^somewhat by*the types of ' teacher activity factors, 
which have negative correlations with attitude outcomes, in second gra'de. 
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Table 186 reports negative correlations of the teacher activity factors 

of Circulating, Helping Interaction with pupils and Available for Help 

with the total survey measure. 0$ th6 other hand, Checking and Listening 

to Pupils Read,, Introducing and Answering Questions, InstVucty>ffby Drill 

and Review 4 and Checking and Explaining Errors (mathematics) were positively 

,correlated^with attitude and <self concept. 

The relationship between teacher activity factors and attitudes at 

fifth^grade support the contention that particular types qf teacher acti- ' 

* 

vities in reading and mathematics are conducive to the development of 
either positive or negative attitude outcomes. , In fifth grade, Giving 
Instruction at Board, Introducing the, Lesson, and Asking Questions about 
Problems were activities with positive relationshifTto attitude and survey' 
outcomes.. On the other hand, the Circulating and Checking Pupil Work 
factors both imply opportunities for teachers to see how well pupils are 
responding to tW assignment with the resultant possibilities of some type 
of negative or positive teacher feedback. These factors were both 
negatively correlated with attitude outcome. 

' Teacher responses . The correlations of teacher response factor scores 
with attitude outtpRies (Table 187) shows a positive relationship between 

i 

t 

the factors Positive Reinforcement and Neutral Feedback in second grade 
reading and the total* attitude at "the end of the year. The Positive Rein- 
forcement factor in mathematics lessons, on the otKer hand, did not corre- 
late with total, attitude, but the^ Negative Reinforcement factor had - a very 
positive relationship^ to 'the survey measure. .This high correlation between 
the Negative Reinforcement factor in mathematics and 4 the t'ital survey 
-measure, ^especially since negative reinforcement was' relatively uncommon.- 
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across all classrooms, suggests that teacher management by Negative feed- 
back to some pupils may assist the others to self-dif f qrentiation and an 
awareness of their more positive behavior. Absence of teacher feedback, 
as reflected in the Ignoring teacher response factor ia mathematics, was 
negatively correlated with the. second grade total attitude mea*siire. 
Teacher response of some type is evidently important to pupi.1 attitude; 
ignoring pupil behavior has potentially negative consequences for attitude 
about mathematics activities. t " x 

The Negative Reinforcement factors for both behavior and academic 

skills in reading both correlated negatively with attitudes of fifth 

t 

graders. It is interesting that thfe Neutral Feedback and Positive 

Reinforcement factors in reading instruction had significant negative 

correlations with the survey measure. The 6nly, explanation that we ca,n 

offers here is that the selectivity of the teachers with respect to the 

reinforcement of particular pupils affects the wgy children in the classroom, 

on the Average, change in their self concept over the year. Even though 

> * * 

reinforcement in fifth grade reading activities, whethet positive, neutral 
or negative, was negatively correlated with one or the other of th^ 
attitude measures, the Verbal Reinforcement Responses factor in mathematics 
had a positive 'relationship, to the total attitude measure* Nonverbal 
Acknowledgement was negatively related, 

Thes£ data'repor ting the results of the correlations of factor scores 
with attitude and self concept measures lead 'to the general conclusion that 
the fidelity of teacher reinforcement of pupil behavior and the fairness^ 
and cor^ist^ncy with which it is ippJied has an important relationship to 
the growth or decline of attitudes of pupils about reading and mathematics 
and-their self concepts. • 
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teacher "behavior you are likely .to observe.' Howej^r, teacher responses 
may* be^nonverbal ahd sometimes the teacher i^iinavare o^the pupil behav- 
ip-r'' or simply cho£se§ to- ignore it. Etonverbal responses $uch as ."smiles'," 
"notices," (without a^^rtic^Cr* facial ex^petssion) "frowns" or "ignored" 
shoifld also b^^nd>e^fed when^approjjpifite in order to have a complete 

v ^^^^^ * • v 5 

recc^^a* the kinds of^ teagjaer interaction which accompany pupil* behavior.* 
?cr some pupil^e^vts, however, recording a teacher response may seem 
le ^inappropriate ; in these cases, the observer will simply indicate 
this by "not 'applicable" or "none 1 "or by some oth^: statement yiescr t ibing • 
the. event in zhe, space or the recoj^L of the teacher response . 

, Teacher Behavior Whic i h--rirrT-ttates a. Pupil Event . , Quite often you * ~ 



are likely zo observe the teacher saying something directly'to a child, 
for ^example instructions to a lesson or a reminder about wha^>the child 
should be ^oing. When such teache'r behavior is directed .at *a target %_ 
•oupil, you should be certain that 'you are making ^ record of it along^with 
the r5upil response^fctx the teacher. \ 
What ?os i t>erfiS hou ld the O bserver Take in the 'Classroom? 

^ * 5 — 7 : T 7- 

The observer will assume a role somewhere between participant observer 
and t>o£te& palm. Each, observer is expSctecl to become a familiar enough „ 
figure to the children that he will not attract unusual attention w^iile 
.entering and leaving the r$om. In the' room, depending on the ^activities 
being carried' on by the children and teacher,, the observer^ should position 
-4fapgglfc_fnE maximum .visual access to the activities . and behaviQ^S' 1? 6f the 
* children. This will sometimes involve the observer with ^the activities^- 
of the 3k\£ldren; sucti ^nvGlv&itent is/^ermisSible so long as the observer' 
does rot tec-jne a teaoher '.s, aide-jyr "take" over the teacher's responsibilities. 
-Acceotable involvement might include a short period of individual work ' /* 
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Instructions for APPLE Observers in the 
Beginning Teacher" Evaluation Study 



Nadine ,M. Lambert 
Carolyn S. Hartsough "~ 

University of California, Berkeley 



OVERVIEW 

The goal of classroom observations in the Beginning TeaflfKer Evaluation 
Study (3TES) is to specify the observable teacher and pupil behaviors 
vhich affect pupil outcomes in learning, The APPLE (Anecdotal Proccessing 
to Promote the Learning Experience) observation, procedures which you will ^ 
* be using have been developed by collecting natural- classroom ^observations 
in over 150 classrooms of about 2000 elementary school children over a 



three year period." 5 " The information about observable pupil and^eacher 
behavior and the Drocedures for collecting classroom observations have 
been adapted to the goals of the BTES project. However, the kinds of be- 

■f X • 

havior that each q*f the BTES APPLE observers will be recording will be 
very similar v to those that were collected in the three year longitudinal 
study. You tfill be using, therefore, procedures that have been ysed by 
many other observers with children of t]^e same age levels that you will 
be observing and in classrooms that will be similar to those in the ori- 
ginal* project 

\ GENERAL GUIDELINES 

Eachl^obseryer working in the BTES project should have some knowledge' 
of the intellectual, social, emotional and physical characteristics* of . 
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childrfen'in the age ranges tp.be observed, second, and fifth grades. Such 
Jcnovledj^g along with previous experience in simple* classroom observation 

^andkfas cessment , are the two primary prerequisites for the observation . 
task. Moreover," objective descriptions of the behavior of teachers and 
pupils are mandatory so that these behaviors can be classified YsA^s^o^ 
ing to the APPLE lexicons. 

. Target v Pupils for the Observafipn" 

Each observer will be given the names of children .who have been ran- 
domly selected from each classroom for observation. Hopefully each teacher 
in anticipation of the observation day will have every child wear a name 

tag so that the target pupils will be easily identified. 

% « 
Classroom Periods Designated for Observation 

There should be no fewer than three periods of observation during the 

school day. The observer should consult the teacher, if this information 

is not available prior to the, observation day, about the times vhe^read- 

ing is taught (usually two sessions a day) and the times when mathematics 

is taught. These periods, plus any other periods that the teacher desig- 

nates as ones in which the observer can observe pupil and teacher behaviors 

associated with reading and mathematics, will constitute the observation 

periods . Normally this will be four 1/2 hour periods a dayj though there " 

may be some classrooms where more periods will need to be observed. 

What Activities or Behaviors Should be Recorded? ft * 

• . . * < 

While the observation record describes pupil behavior and'teacher- 

in* 

pupil' interaction, "it i s important that the instructional role of the- , ' 
teacher be clarified. The observer, therefore, will' nesd'to note the. 
teacher activity which initiates or accompanies the reading or mathematics ' 
lessons. Normally the observer will simply record what-role the te.acher 
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takes at the "beginning of "an observation period or at the time of intro- 
ducing a new lesson. » These roles may include explanation of a procedure 
or a lesson, organizing the lesson activity, leading ar<Lesson activity, 
conducting a discussion, lecturing, or leading a question and answer period. 
Procedures for making these notations of teacher activity will b<? ^described 
later in this manual. % » 

* .* 

After noting the general teacher activity, the observer wL 11 then _ 
make an observation record of the behavior of each pupil on the list of 
target pupils, including in each observation the niandatory information for 
behavior events which will be described later. Following*this initial 
' record of each pupil's behavior, the observer will continue to make obser- 
vations* usually no less than two for each five minute period, and as 



many 



as needed to provide as complete a record as possible of the target 



pupils' behaviors. , _ * 

What Activities or* Behaviors Shouifl.Be Recorded? r 
Pupil behaviors . Almost any behavior of a child or children that 
comes to your attention may be considered. However, we are looking especi- 
ally for the following: / ( 
\ 1. Academic - Intellectual 

* 1 * r * / 

a. , Sill's"" 'of unusual/strengths or weaknesses, including or,al 

» and written vnark, response tt>" question, teacher comment sj e" 

b. Change trait * usual, lev^l of performance 

1 o.c Hes|D0nse to new methods, teacher, etc- * 

d. ^Performance in various situations, including small, 

group-, individual aftd class instruction * 

Social - Emotional . K # . 

a. * Usual method of relating to peers * 

b. Changes in peer relations 



* 
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c. Aggressive,, attention-getting, immature, or inappro- 
priate Jieh^f or ' . > 

' - . , ^ *^ 

d. Successful coping'with a difficult or potentially 
'difficult- situation r - 

e. Relationships with familiar adults - teachers, aides, etc. 

f. Relationships w\th other adults - principal, nurse, 
counselor, substitute* teacher^ etc * 

General Behavior..'* 

a. *^tt^n^ion level , 

b* Restlessness 
y ; 

c. Alertness 

i t 

Physical Appearance (if out of c&rdin 

a. l General grooming p 

b. "\ Health 

c. I Signs f of fatigue 

d. Nutrition . 



p. Statuie (weight' add height 1 

/ s/ 

Consultation about iPupils* 

— - ' ' / ' ' ' 

r/'^Coirtadts with othj^r -school ■ off icials 




y 




acher reports or requests for assistance * r . ; 

' Anything else •you deem iraporfeaaffT Bje„ sure the reason for 
z making the observation' is obviousuto others,; otherwise it 

"will 'appear that y6u have necbrded a trivial event 



Teacher responses Ito pupil behavior eveftts . Since one of the ^objectives 
of the BTES observations is to. ^determine. the nature- ofjbea^^^^^il inter- 
actions, each' pupilt behavior event mus£/be accompanied by. a description 
of thte teacher response, , In the APPLE Lexicon ire have". defined a. -fairly 
large number of teacher responses which have.been observed *tb , accompany* 



^jpupil behavior,/ 1 These descriptions will give you examples of the^^g^jpf £ 



teacher "behavior you are likely .to observe; Howey^r, teacher responses 
may be nonverbal ahd sometimes the teacher i^iinavare oj/'the pupil behav- 
ipr''or simply choose? to ignore it. Ncsriverbal res^nses such as ."smiles ," 
"notices/ 1 (without a>t5articyJ^Cr facial expr^sion) "frowns" or "ignored" 



ahoifld also be 



\ 



ed when appropriate in order to have a c&mplete 



reco^^Of the kinds of_ teagber interaction , which accompany pupil* behavior. 
For some pupil^s^fvts , however, recording a teacher response may seem 
xtte*JLn appropriate; in these cases, the observer will simply indicate 
this by "not 'applicable" or "none" or by some oth§£ statement yie scribing 

thfc. event in zhe, space^for the rec:>j^l of the teacher^ response . 

* • * * 

" , Teacher Behaviors Whioh-rtrrtrfates a. Pupil Event . Quite often you *■ 
k 

are likely zo observe the teacher saying something directly'to a child, 
for- example instructions to a lesson or a reminder about wh^t*the child 

' ' ' • - * \ 

should be ^oirjg. When such teacher behavior is .directed .at -a target ». 

pupil, you should be certain that you are making a, record of it along^with 

the pupil response^, tne teacher. \ 

* ^^"^ * ' 

What ?osit>g^Should the Observer Take' in the 'Classroom? 

\^ w " — X : ~/~ 

The jbserver will assume a rble somewhere between participant observer 
and t>ot*t.e& palm. Each t observer is expected to become a familiar enough . 
figure* to tne children that he will not attract unusual attention Hhile 
.entering and leaving the rrOom. In the' room, depending on the .activities 
being carried* on by the children and teacher,, the observer, should, posit ion 
-4»iinag'iff fn^ maximum .visual access to' the activities . and behavio^g>6f the 
'children. This will sometimes involve the observer with ?the activities * 
of the o^ildren; sucft ^nvOlvSirtent , is^ermisSible so long as the observer' 
j^oes r.ot become a teaoher ^.aide^tar "t'ake over the teacher's responsibilities. 
- Acceptable irrvoivement might iflclnde a short period Of individual work 



> - < • - •. 
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\ 



so [ that evidence; ne§& no% be cited in support 
of this. It is the responsibilidb^ of the' , . 

.observer, therefore, rftot to ^^eralize from _ ^ 
the first .occurrence of a^particuistr behavior. 

Keve^ make .inference^Vabout psycho-dynamic constructs such 
as "reality te^ti-ng,* "body^imagejT "acting otfjt,">etc. - 



J 



* y \^ ^SING TltE OBSERVATION FORM 

The observation as it is actually recbrde'd is called :an EVent . The' 

*V**» ^J^U-r-r jgtsz-oSix 

Ev^ht ^ is ^defined as anythi^^hicj^^-ob^gver^^^ a '^^4 d^v^^rthing 
vhriclTvas jpb'served to happen to the child or which *£*s. said 'of .hipjby' an*. 



>d*lt or child in the school who Knows £im as a ' mejriBer "Qj^^ie^-classrQojii ' # 

v g£bup* or as ah" individual; any betaayior *^f the tlacher ^peciplcally dire*cjjsfa^ 

ttrw^rd^tlie child; o 1 ? a#ditiqn^ fc information such as" family ^tatu^tfr medical 

history,, -In- the mo^tlusuSl' ckse^ou Vili -be writingVa sipgle observation 

( Evfeht ) in'eahh of. ths\£hree sections- of the bfeseK^tio?r page. - ^ ^ 

. . "Please Zrrite /your. dteeiTvatitms in ^felt, Differ abijr blue t or blacH". " Dp v 

not like* either Wncil or red inkV ^enciV tob easily becpmes smudfeed* wipi« ^ 
* * '* < ** **• " * s | * c '/ v *' " ""^ 

'the^handling^your observation, fopa^^ill.re^ive 3iirj.ng coding aa^kejp-. 

pvuichin^, and red ink ^isr^rve* specif i tally .for coding '^ch reiiqr 

'§ince s^ediie- els e^vill,' have to "read wha^t jrou hav« ; written, y.ou Mst^write, 

< legibly, , If your hand^iting Is habitually messy or difficult fro. decipher, 

'^it viil*T>e worse mder%he pressures and incohvenienes.of/ Recording in 

' the classroom." /Therefore, you' may wish to print i•ath^r th^n .to write in 



script. J , ' 

^ Identifying Information 



fiqfer. t$>> the l; sampl^ observation, form. " In the tupped* right, hand ; corner 
' ydu will find ^pace .to fill-in your name,, the date' on. which you are making 

" , . ' >■ ' / . % 1 •/ " 

k the teacher of the/ phi Id ^ children 



you are 



the observation, the name o£ — 

observing, aftdr the, grade, level at.wiiich ybuWe observing.. Yqu will use 



groub to a selriJ-worfc assignment.' FollcvLru ru.eVjf mking an observa- 

tioa for lr each significant behavioral event olus e^ery change in a 

pujd^-'s learni&g context will -provide^j^j^-etfsoj^e of ir^fortnation about* 

pugxT benayio^s in reading^anjL nat&eraatics alon^Atu t,ne teacher inter-. 

actions union- accompany t>~ese pupri oer.avicrj. ^ : _ » 

^HoV" gho&ld Pupji Activities and Learning Contexts ce fiecorded ?y 
J = 1) r" ~ ~ ; ^ ; : : ■ 

. - Bsrch "cisie :;rf£ observer nates an evens, N tne observation form prcHPTdes" 



^^e^^^r^^iie Observer Specifier tae HfPIL ACTIVITY ^nd CONTEXT r for that 
event/ ^ !Iescfr4^~i'oii of each of these aspects if^wtseryed behavior, 'and 
the labels rto be used c&a^be found^ih the. se^wiaift b* -this * Manual entitled 
USING '«H£ bBSHWATION-POafi. - " 




O » • • - INFERENCE IN OBSSRVM'ION ■ • 4 c •■->.• 

„• • *: - - • * * ' a * " •- 1— * 

How Will Yosa Maker Clear 'the Intent -of Youy- Observations ? * ( : ■* ,» > 

'^The behavior which is^ecorded^may be considered, uO^elong'Ojx a con- 
ti;iuusr,of descriptive precisioa ranging -frorf s i mp le/^s t at etne n "ts.abou t overt 
acts , ~ ,f D4vll put his 6oat on n to inferences regarding the 'intent or meaning - 

of •a" behavior wi;tboux re.f e't'ence tp the act itself,, *Dayi.a is ^feeling + inse^ 

C ■ * - : . • * c ^ 'ffc - - (| 

, ^^lre. , ' t, ^^laKing inferences in observational p^3cedure,s 'is unavoidable When*-, 

v . , " w ' C , 4 * - # - ' • 

everyvOyL choosey to record a par^cu^ar^ccurrencej.you iiave ^inferred some-; 
thing about "ihe Behavior of the crfiid vhfch claused ^ou to jeaajie a refeordT 



Sonftktes^S' of inference 4i?e tie iter* thaa others, novever*. and^you will" usually ^ n " 

• * * - v.. ^ *, M ^ ' ■/ ' 

vant to\b^ somewhere in. i^be fiddle ^fi^hp conti'jaiiyia by adding, import ant ' *' 

facts 'Which struck #our ^ttefition, rather tlian at either extreme . \Fo$' ' • ' . *J 

* / • . ' * * ^ ' * s r ' < \ > -ft" ' v 1 ^ ' ^ 

"eic?^ip\e .thirf Denervation red&rd describes What <?>j.e child did^vh^t v the» G^a,s,^- % * 
iro'om cir5Ui»staofe^ s at t^e 'time and yWt ^inf erep^le yo^i made, about thLe/,-* * * " „ , , ■ • 
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given, ehough accessory*inf orroation is -provided to identify" ^the act of* *" 

David putting on his coat' as unusual. ..Tire observer makes an inference and 

~ ■ * % C , ♦ 

later this inferenpe is ident%fed as* such, in the »ata processing records 
X^ here are some types of inferences you -will want to avoixL- however 
as well as some you WilKneed to make explicit/ Three "types of* explicit 
inferences.' y#U might make coiiia be statements about: 

* . . , ; , ■ ; 

1. Possible causal relationships 1 v / * 

2. Inferred personality characteristics. ' ' ^ f * 
• " • * „ : * , ■ . 0 K * . , • 

V 3- Relevance of observation to evidence o£ a stressful" behltvi^or 

or clas-sro&n situation/* * , - - . f * * 

1 When you' do make 3uch inference?, plea.se observe the^e- rules;/' ' 

1/ Do rtot embe^ryouii infter^n^e* in the*~obser,vation * ' " v ^ 
- ^ ' .-V itself v -Separate "the ^Inferential statement % 



"BAD:V^ Don/* 4 e P enci fttt child, cried when *his'-^ 
- .^ther .lePt the room^ . /; *{\r v t. * 

BETTER/ • DorhsQried when his ^mother left v the rooi. 
^ m .i (Probable sign of dependency.)/ 



2^> *Wh§n the^infefenc^l^ire ff^d^tfi^ob^ervatlon is labeled 
- 1 ion tfte basis ©f t heTJn^Tem^^at he^ than -the bocTy of ,_ t 
\ '-the observation. In- .the above example -this is impor- 
«« tant because "the /ir.st statement should probably £e 
called CRYING vyhi\e the second shou ±a~fte labeled 
•I)EPEND$NCY NEEDS, It is as6me2h^hdt s in this case - ""; *' * 
'the observer considered the,' dependency Behavior more 1 
important "than tha crying.. ' , ^ ' 

/ / * /*> r £^7 \ • - * 

.3. J l3o' noV-generalifc^ from a single observation. . a 



— — - - 



1 / 



EXAMPLE: 



Steve hit,Pe : < te .without "apparent 
provocation-; ' X st ^ye has a L lar,ge. 
resefvoii* of -hostility ) . 



In: th^se^an^le we' assume that the observer* 
" thas seen examples of Steve f s hostility ^bef ore. 



a 
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so "that" evidence; neejj not be cited in support 
' of this. It is the responsihili^ of the 1 ' 
' .observer, theref qr e , mot tb^g^neralize from „ 

the first .occurrence of a^particul&r behavior. 



rv 



Never, 



S? as 



erT make inference'SVabout psycho-dynamic constructs such 
"reality -testing,* "body^ImagejT "acting otft ,".»etc. - 

y : -\ / /USING TlfE OBSERVATION FORM . ■ " * . / # 

The pbservfeionv as it is actually recorded is called ;an Eftent . The" 



^ EyarrtT is ^defined as^S^hi^^hieJ^^obsggver;^^ a 'clu^ d^^^hing 
wkielTvaS otfser^red to happen to the child or which #a*s. said 'of him by 'an^ 



dtlt or child in the school who Knows £im as a'ngmller ~Q:f^>he,-classrQom 
*g#bu$ % or as ah* .individual; any behavior is>f the teacher specifically direct 
ttw^rd^the -child; ■ tfr a#ditiqn&T information suck *as" family ^tatus^w medical 

history- 'la Jthe nogtlusufil" ckse^c&i Vili /be writing^a siji^te qbservatio|i 

W/ *~ * ^' * ' "r - - * 1 - 

( Evfent ) in" eafeh of th$\thre$ sections- bf th,e bbsesyeftiogr page, v ^ v 

/ ^P^ase in ^f^^^eferabi^r .blue^or ^lacH". ' Dp" Y 

not^use- Either trencil or red inkV *P^cl> tob easily becpmes sxradg&P wtfcfo; 

the* handling ^our observation fopr^ill.r?oeive during coding a^^keyt-. 

punching, and red ink is r^Wrvebr specifically .for coding *e£ch ^niit^s recqr 

. Since spmeojie* else^will/have to -read wha<t you hare written, ypn ^st,,write y ' 

c t ' ^ • JL* " . \ • * K 

- legibly. , If ypur handwriting Us habitually messy or difficult /to. d^cipfter, 

*it will*T>e worse tmder^he pressures and incofivenienes^f; Recording in 

' the classroom// Therefore, you' may wish to print rather thsn ,to write in 



script. 



' ^I deQtifying Informjation / ; 

r — - . . ; ? , : ; 

V Ref6r to, the . ssmpl^ observation. form. ** In the/'-uppef* right, hand, corner 

■ •* ■ ^ • - ! - • * / 7 f 

% ypu will find space : to. fill" in your name,, the date on. which you are making 

4 , 1 * r . ' 'r- ' / \ / '/ ~ J 

. . the .observation, the name o£ the teacher of the^ child „qr dhildren you "are - 
, observing, ^id^the grade level at.wiiich -ybu^^/e observing., Yqu will "use 
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Ai?brevia^ ^ Context 
, ; tion u Name 



TRANSI- 
TIONAL 



Transitional 



-14- 



Def init>on 



Times which are transitional between class- 
room or playground activities. ~ " ^ 



CLASSROOM EXAMPLE: Changing activities 
changing froia oral to Silent reading 
group, getting ready for lunch. 

PLAYGROUND^ EXAMPLE: " Lining up to come 
inside, changing -games . 



TiST 



Test 



Tiates, either in classroom or on the 
playground, during which tests are 
given by. the teacher. 



* - Start 
CONTEXT. Time: £nd 



: It is important in the BTES observation 



re — ri that the Observer 'record the duration of each context fo^ a particu- 
lar' subject. Ordinarily, the same context continues through- an observa- 
tion periodk however, there are times when thd teacher dismisses '.a- read- 

i ; V - 

ifig groupv^ps example, and .the children move on the their desks -to^undertake 



a new L 



r le^rnirig activity. For -each initial, .evetot* for a .tarcetedr pux>il in ' 

. \ - 7 rf r J # - , . . * \ 

Tsne beginning \of an 6bserya?tiQft' "record, ,tfi«* observer should record' the : 

time o* the beginning of that 9pntext. * If fche^ccwtext changes, it , 

* \ * •* V s • * - ■ i% * . T ' * . . ; \j 

mandatory that tt\e observer- make a record" of a' flew pvent 'for a pupiT • *- 

\ ~ ♦ ' / r 

indicating, the newWntrext an* the tdm^at* «h4ch it begkh.! The2e><ime- 

\ , - / , * < * * .* >' <- 

records may.be made'dWy once -for ea^h v chllji at* the beginnipg ,ok an t ob- 1 

servation peg&od ,-but - Additional time 6ntrifcs»yill accompany each new * 
. - ~ <" ^ ° \ /J" / „ . />' % * 

context . At t£e" end of "Voe^^bsBrAtion period the 'observer ckn -fill in"' 

the ending time iftfr eacrr time entry 'yhich m&y be simply the tim^t whi£if~ 



the observation -Deriod .endecl. 



- CQDEt The code referred to here is th£ BTES ID code assigned to 
% > - * ; *' 
the vuoil. • This will be found on the list^bfi target pupils you will 

receive prior to your observations.. Put this.- number ' in the box, f . 

SUBJECT: / The r fV^t\mandatory eiitry is SUBJECT This refers to the^ 

pupil in'the observation 'being recorded. Although there may be other pupil 

*" v * ' ♦ _ * « ' , ' 

in the observation, ttye.JStffiJECT is the. principal pupil in. the event. , 

PUPIL ACTIVITY; This entry refers to the content^ of v the pupil's 



instructional or cla^s^^ny^^^ 



is noted, ^ A list of frequently used activities is provided below," These , 

. ' J \ \.\ ' ' * - " ~ . % ■- 

'are suggestions for what might, be recorded. In &11 cases; make as - 

'sp^icific record as you «can to describe the child's learning activity or 

«. - > • „ , * * 1 *.* 

other classroom' activity. - 

J CONTEXT: The entry CONTEXT provides $. description of different 

-school settings which may Kave important influences on the • child's 

school tehavidr. Context *is» determined not by the event itself but -by ^. • 

the school situation' in which the eVent occurs. ' Centers .designated T 

"are, those situations in which the teacher' is directing the activity.. ' ' 

Contexts designated A are tho^e in-whieb another adult such* as ^l.aide^ 

or voluiitee^parent . is involved^ Tho|6 designated I_ -are situations in % # 

^vh^^ in&egettdently or the teacher's direc-* 

tions. Explanations 'and examples of the several 'different CONTEXT 

entries follow, " -v* '* - 
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Abbrevi- 
tion 

T- CLASS 
A-CLASS 



Context 
Name 



Teacher-Class 
Adult-Class . 



Definition 



Teacher err other adult* i<s directing 'the 
activity and the entire cl&s.£ Is 
pgRiciibat^tg ' ^ ' f * <' 



a « a 



A-GROUP 



Teachers-Group 
Adult-Srouu +<r 



CEASSJlOOft EXAMPLE: # Flag, js^lute, stoipr 
£i&e, sharing, directions; *iess*<2ns.^ 

, PLAYGROUND' EXAMPLE: Gabies directed by 
_teacher -,in. which tfkole* class is. parti- 
cipating, • w - r r " - ? 



The class is divided into groups' and tjhe 
£eache> or other, adult is/direeting^ the •• 
. activity o^ the group Jin ' vhiqft *t<hfe chi\cL 
for whom the '.observation is redor^lecL >s' < 
participating*^ . / . f ' 



, v , .£Ij£SSRO0M_ EXAMPLE : Oral reading groupf, 
i fl^tji group. " > . / 



h PLAYGROUND- EXAMPLE: • .A game under the 
| direction, of >the/ teacher but not in- 
. ■ \ $ vc&ving t'he entire^ clagi\ , ' . " ^ , 



» 4 



T-INDI.VID ^acher-lndlyi^ - -$he .teacher. ds- other adult* Is directing 

A- INDIVID - dual * ' ^he activity, f af an individual^ student „ 

*' Aduit-l'ndiviV * 'and not t th^ rest of th£ % ~class. 

• dual, > ' . ' 



f ' i >'CLASSr6gN* EXAMPLE': . The teacher has \aiie& 

„ •* J a ^tu'dent to "her .desk for ^peqleci help - v " ' • 
1 4i 5 a# observation' is recorded for him 1 

^ , ;-during this tim£. {'-'*' ' 4 ' 



: ERIC 
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Abbrevia- 
tion 



Context 
Name 



T-INDIVJP ' Teacher-fndivi- 
' " dual 
A-INDIVID . Adult -Individ 
dual-. 



I-CLASS * Independent- 
Class* ; 



\ 



3-GPOUP 



Independenc- 
es dup* 



PLAYGROUND EXAMPLE: '.The teacher~is' . 
giving instruction 'to a single child 
about a game and an even£ is recorded 
for *chat -child during ohis -time. 



The entire class is working ^or playing 
independently of the teacher 's-direction . 
Teacher* usually present ^but not directing. 



CLASSROOM EXAMPLE: Math- workbook* assign-' 
merits^ silent reading assignment:. 



PLAYGROUND E^MPLE: Free play during the 
period dev orbed 'to physical education. 



A. group of students • iV working or 
piayd^gMndependently of the teacher's 



&i\C&£tion. 



\I-INDIVIp- \ Independent - 
' Individual" 4 



ERIC , • 



CLASSROOM .SX AMPLE: A group of students re- 
working on, independent reading assign^ ^ 
ments . (workbooks) . r Npte ; »TJie teacher 
will probably be "instructing another . ^ 
, group during xhi-s" time,, "An event"" 
recorded" for chiM in the instructed group 
would be l v abe,led T-Gruup r . .. ' * 

PLAYGROUND EXAMPLE: 4 A gro^has formed . 
spontaneously or has been' fornie'd.by the' 
.teacher and .is now, playing "free from the 
teacher's direction. 



An indivrdjaal student is wording or/playing 
independently of /the teacher \s direction, apd 
also -independently of ,the rest jtft the* class. 



• CLASSROOM EXAMPLE: v An;, observation is 

recor-4^ f or a e< student who* is* working alone 
' on a programmed" reading .activity; any type « 
-•o£ individualized activity which t!)e cjuld 
performs .primarily without teacher direction,, 

PLAYGROUND EXAMPLE*:'' Ttfe student is alone 
" on, the' plftygv Qund * studenit is practicing 
alone ^oipe physical education skill. * 

_J : A 1 : > ^ ^ : 
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DESCRIPTION OK 'EVENT: You now have 3 lines in whibh to record the 
actual observation.. Please follow the few guidelines below when writing 
your "observat ions. They will be easier for you to encode and easier for 
the keypuncher to transcribe if you do* , 

I t . 9 Record the sequence of an event in chronological order . 



BAD: Allen and Steve were reprimanded 
by teacher for not being quiet 
during spelling lesson*. 

BETTER: Allen and Steve were noisy 
during spelling lesson. 
Teacher reprimanded them. 



r 



2. HEVHB make a running commentary on classroom activities . 
Separate -the anecdotal material into events which can 
stand alone. If hn incident seems to consist of more than 
one events still record each probable event in a separate 
^space. " . , ' 1 ~ • , • * * 



3/ Report 'the circumstances surrounding the event in sufficient- 
detail, including .precipitating factors » prior* ^condition's^ 
' consequences , follow up, etc . If tfife event itself 9t complete ,* 
with all tTfie. surrounding information , is\±qo lori^tp record - 
in a single 3 'line s^ctiorr^ yog may Continue it dn the 
following -section,' indicating' ^haf it i£ the sam§. event by* 
. scratching ou^.the' horizontal -double 'line. TEqu, may find - 
< " that in jreportin^a^aptlcqlstf incident in sufficient ''de- * . 

.^tail you will jhave* more than one event. In thfe. latter 
{ ' . "case be-'sure to. separate the events on the ^observation 



- TSACKEft- ACTIVITY : The labels: for IfiAGHER ACTJVXTL.in -the- space at ttfe top 
/ of .each observation record willfbe completed ^by the project -stair. Ort.the^ line 
' opposite /iTeachjer act&yity" , the ohs e rye r- should" describe .the teacher's activity 
^Vhioh "accompanies the. pupil event. Normally this indicates, simply the role of , 

V * v. * * " ' } * > ' 

the t'feabher' during -the "observation .periods Jhe following list of roles' will ^ v ; 
give the observer an idea o£ what should be recorded iV, this line* , % 



asoS 
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Examples, of teacher activities: 

Leading a discussion * 
Lecturing' 

Explaining a process, procegxfe or a -lesson 
Providing information a£#trc an activity or less 
Org^nizin^ an activity or lesson 
Conducting a" question arrtl answer "session 

Leading a game- %> * . " 

Leading an activity* \ 

Reviewing the work of a group of pupils^*? * " . , " 

Reviewing the ^work of" in dividual pupils * 

Standing," waiting for silence * ^ .- : * » 

"Calling class to" attention ' " *~~ ' % " 

Moving around the room* assisting children with, assignments , 
Available at d'&sk"to help individual pupils 
At desk working, relatively unavailable to pupils. ^ . 

Teacher not in the room ^ 

Conducting a visual ^dembrist rat ion^ (models , pictures) * * # s J 
Conducting an audio-visual demons tfr&t ion # » (movies film strip.) 
Conducting an audio de&on'st ration (records^ tape recorfltiri^) 

» » > 

The- project staff jfill" develop the le^icon^ of teacher a#ti^ties on the % 
basis q£~tne records of <ffhe otrserv^rs; It is therefore, important ".*to . 
'describe the teacher activity as specifically as -possible* Nortrfially the ' - 
teacher * 3 activity, like the context will not change oftlnL during an>fob- 
servation period. In order to* save time duririg observation periods, the 
observer ,may wish to make a record "of 'the teacher activity once to keep 
track 'of. it, arid, then fill in the space "for each event on the observation 

* ' ' 1 ** > #. * • <? 

v - . * 

record after the observation period is over^ Only changes in teacher 
activities need to-be recorded, .but teapher activity should Be as specific 
*as possible to the pupil rLearning *an/l behavior events. * / 




• 3§J 
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TEACHER - H£;3PON53£ : The labels* for TEACHER RESPONSE will be re- 
-corded by.tlie project staff in the space at the top of each observation. 
The .observer may wish zo refer to the Lexicon for examples of teacher 
behavior ^rhich have been observed in the de^ei^^ient^f the APPLE obser- 
vation system. The observer will note that many of these Lexicon terms 
describe ways teachers reinforce, positively or negatively, pupil behavior* 
in the classroom Other kinds of^te'ach^r respond are also defined. Fpr 
each pupil event the 'observer must reco rd the teacher response on the , 
appropriate line.. In .the^case where a teacher behavior' ^initiates an * 
event, the observer must also record -tfoe pupil ,^esponse^. _Ih the* case wlie? 
the teacher^response* is non-verbal^ -it i's' important that the appropriate 
description of non-verbal benavio? be made*:,- Frowns, smiles,, noticing 

- v » * • 

but not cqpmen£ing, rewarding with lifesavers, ignoring UeRavior, .or un- 
able to observe behayiof , would oe examples of nbn-Mrer,bal teacher be- 
havi or which smight a£cQmpany a ^pupil *event ; . -Please.-^e as. specific as 
possible in making a /Record, of tfre.-teacher Response. If there is, no * f 
^response or_ no way to make a^record of the teacher .response . put "none" in 
the space opposite "T&acher Response." * , i 
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^UAiTtTY OF PUPIL BEHAVIOR: iVt-his space for each observation the * * 
► e ^server °is asked to rate h'is 'judgment of the pupil 1 s behavior with respect. > 
- .to'the classroom activity taking r>lace> a± the, time of the observation. " \' ' 

"'v ' • < ' \ . : " * ^ 

• A 1| ^ f ' rating would be assigned to -positive, productive, involved behavior. • "* . 
• A "- ,f mating would be assigned to an event in which .the pupif "behavior 'was. \y 
- -ndn-prpduct^Ve,, in, which the- pupil failed to*unde£stand a" concept or % 
. failed to* get .an .assignment . 'A fr 0" "would he used when the quaiity of %he. 
aehavior is indeterminate.' 4 . ; \ " ^ / , . (/ , 
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" "\/ * : . COMPLfcTltfg THE OSSEByES' SUMMARY REPORT ' T~ - 

/ " ' ' ' - " \ - * . « - ^ . * s -- * . * * 

% ' Purpose of t'he Sufanary SeporV ^ ^' ^y.^T^ * * -V v *. " 
• . Many teachers pai^^lpatifig in uhe'^JTES project. have requested the 
opportunity to^d^c^sr~the 4ay f s^btivlties W^^^^^servet at the cpn- 

* clasibil ofais> observations. The -Summary Observation Report -was pre- - * 
patfsa feb pr§yide a systematic method for -reporting back to teachers- 7 
?fiat general, things were observed. in- each observation period as well as 

* f .'for' checking with '.teachers t&/b£ sure' that no important information abotit 

" '-.the target" rmpiis 1 learning experiences- have been omitted, , , . ^ < tj 

* . . De scription- gf Entries- to be maftg on. the; Form . 
, ' ' ** v The material to be 'completed at «the top' of the 'form is self-ex- ~ 
-."Dl&natory/ Note -that* there* are .three sets, of blanks provided for the 
* " dai ly* attendance record^ot the class .'and target pupils. At /the leJ^ ' \ 
sicLbvOf the- cage ar^* spaces Vindicate th$, class enrollment by, 'ethnic 
""Status, "After* this,, column is a set* of< blanks to enter the class at ten* 

* - r - dance, record of t!ie aay df 'the.,observatf6h'. B At the right side of the form 

v. ' y . - ,.<•;; % *.\* \- 

~ * arfe soaees in whiah th'e'names^Of tiie, children to be observed are entered. 
\«W following each's^ace, is a M P H fio be caroled if tiie child wa^'present v ' 
and" an" "A? to fee' Circled if "the child was absent. Unless a child leaves." 
"during the. day of . the observation, this attendance record would be the 
\ 4ame on the , the' obs,e<rver ''Summary Report for eaclj^gbservation period. . r 
' ' - ' ; Ne^ the observer completes fhe observation summary. At the left 
d ' of ths siSamary table are spaces for the child's tlame and code number. , 4 
\ /This should be available, from thg list of target pupils which you wjir ^ 
have prior, to ttte observation, Next is a section headed M F*or each ,6))-^ 
served' pupils d/ring observiatiofi period VJiat was." Under this section the 
" ' observer should record* the priraary.^UPIL ACTIVITY listed for'thp. 
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observations he made for each target pupil. In the next column the 
observer records the materials that the child was using taking" care 
to, note the name of the book and the page number If possible. If more 
space is needed the back of the foi*m car; be used. Under the heading 
"Focus of Learning^ the observer should indicate ip a£ much detail as 
possible what the specific learning activity of the child was during the 
observation period. t -This is an amplification bf the "primary classroom 
actively" column and provides the bbserver with, a chance to ask the teacher 
to state what specific reading or mathematics concepts or skills she was 
x expecting the child to learn. Examples of specific reading skills might 
vbe "beginning .consonauts," "ending blends ," "making inferences,," "speed 

\ • * "» o 

reading." Examples of specific mathematics skills might be "zerp dif- 
^Cicuity in two place multiplication ," -."long division," "the 9's jnultipli- 
cation rableV^^I^^'ti^ }£itn concrete objects for addition with Sums 
to 20." OT^ousl^the"strstce is cramped for a long^description, but try' 
to write jLn ^etfbugh so that it is po^4^ble^tt5' determine the way in which" s 
the teacher describes what she is hopi^^th^Qhild will accomplish in 
che readirig or mathematics learning. 4 ^/ 



, The next ^sectioa, refers to >a judgment 
tent of the time ouring the observation that 




^occupied him. These activit 



.engaged in whatever classroom acti^ 
be ones which the teach^V directs primarily, or they can be activities 
which the pupi], initiates" \such as undertaking a special assignment when 

s \ \ 1 "S* 

classroom work is finished. \ It will be impossible to make an exact 
time Estimate for this £ntry; however, make either- a "number of minutes" 
6r a "percent of time" estimate on\the basis of your observations, 
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dieting this as soon after the observation as possible would increase 

v * c 

the ajrcuracy of the time estimates. * 

Ofte final two columns refer" to the. nature of the learning aqtivites 
during the observation period. It -is important to determine the extent" 
to which learning activities were structured for each child. An ex- 

ample of a very structured learning activity would be a programmed reading 

r> 

lesson," or a mathematics workbook. An example of an unstructured 
activity .would be a free rbading period where a pupil selected whatever 
he wasted to do. Remember that this is an observer rating and tha^ you 
will have to use your best judgment r r«^+.o the nature and extent of the 
structure of the learning. The final column is used to indicate the 
extent to which a Child's learning activity tfas the sage or different 
from other- members of tne cf&.ss. , . - 

Discussion of the Observation Record with the Teacher 

You and tHe teacher may Ijave time to review the ^obse^vjation record 
toward the'end of the d$y. It. will be more appropriate "for you to^are 
the summary report -£han the* .individual observations. The teacher can 
. assist you. to determine the. nature of -the material and the -focus of the 



leaming,actiVity. The .teacher's knowledge of the assigned work in 
class should assist/ in making -.ratings of structure and differentiation. 
The productive time estimates wili probably, be yours /however, and can 
be shared with the« teacher for her ^information. 

If- the teacher, makes important comments about any of the pupils 
during the review period, tile observer can make an observation record of 
them. They will be encoded, later like hie rest of the observations .and 
nter the pupil record. Examples of comments yhich may be important 

wi\l be thosd which describe changes in putfil behavior sinoeTthe begin- 

\ \ " * '-■ * * '• " I ^ 

ningW the school year or clarify the meaning of behaviotf which' occurred t ; 
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during the day. These 
sample observation recorld 
can be, retained on either 
Report . 
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opport'unities for ^ teachers to contribute to the 
are important ones ancj the content of them 
the Observation Form or the Observer Summary 
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READING ACTIVITIES 

Assigned silent reading' 

"Assigned writing 

Comprehension-literal* facts 

Comprehension - main ideas 

Creative /writing- fc 

Dictionary skills 

Dictation 

English 

Grammar 

Oral reading 

Penmanship 

Phonics 

Practice spelling' 
Punctuation * ' » 

Readiness Activities 
Reading workbook - skills 
uns-pecif ied / 
Recreational reading 
spelling 

Spelling workbook " , 
Syllabification w . / 
\Jord meaning * < 

. •/ 



MATHEMATICS ACTIVITIES 



J 



Addition 
Basic facts 
Decimals 
Division 
Fraction 
Geometry - 
Graphipg' 
Measurement 
Money * 

Multiplication 
Percent, ratio 

proba6i*lity 
Relationships 
Subtraction 
Time 

Word Problems 
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ACADEMIC ACTIVITIES other 
than reading and mathematics 

Anthropology 
Art 

' Book Report-,. 

Colors ./ , ' * 
Drama, ^lay : 

Film Strip',' '~Mbvie 
Food Study - 
Geography 

Physical Education 
Shapes and Forms 
Social Studies 



SKILL DEVELOPMENT 



TRANSITIONAL ACTIVITIES MISCELLANEOUS ACTIVITIES ^ 



Blocks 

Clapping ' ' v * 
Clay, \ . 

Coloring ■ , ^ 
Cooking, * 
Crafts 
• Dan £ ling *• 

Freeplay and. freetime . 
Games ^ • 

Marching ' ' * 

JBody Movement 
'-Music! . 

Painting 
•Plajrh^use 

Recess' V ; 

Records 

Rhythm 

Singing 

Speech k [ 

Story \ • 

Tell~a-story 



Transitional 

Clean up 
x * End of the ofay 
\ Line up * 
A Qtfiet 

Rest 

Snack, #i£k— t£me 

i • .) 

, OPENING ACTIVITIES 



Pledge 
: Roll-call 

Beginning of day 
* activities 
.Sharing 



' Assembly < 
Conference 
Discussion 

♦ Election 

• Individual Activities 
fc Library 

Lunch 
£"arty 

Weekly Reader 
Individual activity 
done at a desk 
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EVENT NAMES 



ACADEMIC PERFORMANCE, ABILITIES* 
AND. INTERESTS 



ORIENTATION TO INSTRUCTION 



Ability . 

Academic* Performance . 
Activities 
Concept Learning 
Convergent Production : 
Divergent Production 
Evaluation' 
General Knowledge 
Interests * 
Jntellectual Functioning 
Language 

Listening Comprehension 
Memory 

Jtfumber Concepts 
Or^T 'Reading 



Phonic Skills 
problem Solving 
^^-^eadi ng Comprehension 
Reading*. Vocabulary 
Shapes and Forms 
Test Performance 
Vocabulary 



-SOCIAL INTERACTIONS' 



t 



\ 



Antisocial Behavior 
* Care-Taking _ 
"Competition 
Conformity 

Controlling Beha v> m 

Copying ^ 

Fighting 6 

Imitation 
' Meddling 
\ Place in Class 

Play Behavior 

'Physical Contact 

; pupil Help * 

Rivalry 

Sexual Behavior 
skyness 

^Social Relationships 
Speaking 
Tattling 
• Teasing 



j 



Attention-Getting Behavior 
Attentiveriess 
-Boredom 
Cheating ^ — — 

Conduct " * ' - T 

Cooperation 
Daydreaming 
Direction-Following 
Disengagement 
Disruptive Conduct 
Distract ibility 
Diversionary Tactics 
Engagement 
Egress p 

Failure to- Respond 
Impertinence * _> 
Inattention 
Participation 
Punctuality 

'Questioning 0 
Reassurance ^ 
Resistance J , 

"Sitting 3ehavior 
Student Wish- - 
Talking 7 * 
Volunteering , 
Wandering *o 
Work Habits » 

AFFECTIVE BEHAVIOR - 

Affect " • 

An&er 

Bizarre; Behavior 
Cotaplairfing . 
Crying 
Enthusiasm ' 
Fantasy 
Frustration 

Giggling . * • 1 " 

Hurt y Feelings 

Nervous Habits 

Sulking ^ 

Tantrum - 

Whining 

Yelling 
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EVENT &AMES 



PERSONALITY TRAITS AND ATTUTUDESf 



i 

l 

PHYSICAL CHARACTERISTICS AND ACTIVITY 



Adjustment- 
Anxiety 
Attitude ' 

Decision-Making -» / 

Def ensiveness 5 

Dependency 

Fear of Failure 

Independence 

Maturation 

Mood • 

Persistence 

Personality Characteristics 

Personality Type 

Possessions 

Responsibility 

Selp- Awareness 

Sexf-Concept 

Sex Attitude 



'Activity Level 
Attire* 
Auto-Erotic 
Eating Behavior * 
Fatigue 

Handedness 4 • , 

Hearing 

Hunger 

Impulse 5 " Control 
Motor Beftavior 
Neurological Factors, 
Organic Signs 
Physical Appearance 
Physical Coordination 
Restlessness 
Sleep .Behavior 
Sucking Behavior 
Toilet Sehavior 
-Vision ^ 
Visual-Motor Organization 



FAMILY HISTORY AND CHARACTERISTICS 



HEALTH AND ILLNESS 



Alcoholism . 
Child Care 
Conference 3ehavior 
• Death 
Divorce 

Education History 
, Family Background 
Family Concept 
Family Constellation 
Family Relationships' 
Family Situation 
Foreign Language v 
Marital Status 
Occupation 
Over .Protection' 
Parental Pressure 
Parental "request - 
Place in Family* . . 
.Race 

Rejection 
School Support 




Hospitalization 

Illness 

Injury 

Medical History 
Medical Treatment 
Medication 
Nosebleed , 
Physical Complaint 
Physical Condition 



ADMINISTRATIVE-OTHER 



Administrative Intervention 

Adult ManageAen\ 

Attendance 

Observer Judgement 

Recommendation 

Scheduling 
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TEACHER EVENT NAMES 



TEACHER STRATEGIES 



TEACHER ATTITUDES 



Discipline . , 
^ignoring * 
/ Individualized- Instruction 
/ Inhibition of Communication 

Instructing ; 

Physical Punishment 

Redirection 

Seating ' f 

Teacher Acceptance 

Teacher Criticism- 

Teacher Defeat 

Teacher Encouragement - 

Teacher Help 
' Teacher, Intervention 
^ Teacher Management 
'Teacher Mediation 

Teacher Praise 

Teacher Reassurance ' 
\ Teacher Reprimand 
teacher reward . 

Teacher Warning ? 



^3 



Teacher 
Teacher 
Teacher 
Teacher 
^Teacher 
Teacher 
Teacher 



Affect. 

Change 

Coricern 

Confidence 

Disappointment . 

Judgment 

Resistance 



.TEACHER COMMUNICATIONS' 



Parent-Teacher Communications 
"Teacher Complaint 
Teacher Information-Giving 
Teacher Request ^ 



